Design engine for automatic layout of content

ABSTRACT

A three-way separation of information in a document includes content, design and media aspects. This division supports automatic rendering to multiple forms of media such as print, Intranet, Internet, and OLE embedding. A method adds content to a composition having a chosen design and automatically calculates a layout for the composition. A content object is dropped upon a receiving component of the composition. The dropped content object has at least one potential type. A potential type of the content object is matched with a compatible type of the receiving component. Next, the design tree is modified to incorporate the content object, and media layout values are calculated for each content element of the composition The dropped content may change the existing content, may replace it completely, may be inserted into the existing content, may change its properties, or may result in new components being created in the design tree. If content is dropped in an incorrect location, a tree structure is used to determine where to place the content automatically. A content drop table data structure is used to assist in dropping content onto a composition. When a content object having potential dropped content types is dropped upon a component of the composition, a matching subcomponent type may be determined by reference to the content drop table.

FIELD OF THE INVENTION

The present invention relates generally to the processing of data by acomputer system. More specifically, the present invention relates to theautomatic layout and formatting of content for a particular design andmedium.

BACKGROUND OF THE INVENTION

Since the introduction of PC-based layout and publishing software in thelate-1980's, the explosive growth of this market has underscored graphicdesign as a vital component of effective business communications. Theinformation explosion has resulted in hundreds of messages--manyenriched with clip art, custom illustrations, and scanned images--vyingfor attention across a wide variety of media (print, electronic,multimedia, etc.). Businesses have a significant need to visuallydifferentiate their documents above the market noise. Pages thatemphasize content with minimal attention to formatting, illustrations,layout, and visual appeal, will no longer attract the attention ofincreasingly sophisticated business audiences. Business users competingfor readers' time now place a premium on creating and publishinggraphically attractive communications.

The proliferation of software tools, ready-made content (e.g. on-linephoto libraries and CD-ROM disks of clip art), and desktop peripherals(e.g. color printers and scanners), have made attractive content andoutput affordable for anyone with a personal computer. In addition, theInternet has had a profound impact on both the breadth of content anddistribution opportunities. However, producing professional-qualitymaterials for disparate media, including the Internet, requires designexpertise, a variety of authoring tools, and advanced computer skills; arare combination among most business people. Home and small businessusers in particular--already taxed for time to cover multiple businessroles--seldom have the time or the tools needed to enhance their designand computer expertise.

To assist in the creation of materials, a number of word processingprograms, desktop publishing tools, internet authoring tools and thelike have been developed. Many of the word processing programs allow theuser to include tables, columns and rudimentary graphics within adocument, however, they are typically not designed to easily facilitatesophisticated document formatting. Desktop publishing tools can assist auser in preparing and formatting more sophisticated documents, as wellas outputting the document onto a medium such as a printed page, acomputer screen or other. However, current software publishing toolsstill require that the user manually change the layout and formatting ormodify the content if the document needs to be output to a differentmedia. Likewise, if the user wishes to change the design, or wishes tochange the layout and formatting, the user must often manually modifythe content of the document in order to fit the new design or layout.This manual modification of content or layout is time consuming andtedious for the user.

A variety of word-processing, desktop publishing, and Internet authoringapplications claim to streamline document production. Each of theseproducts partially succeeds, but with significant limitations. Forexample, current software publishing packages allow a user to author adocument once, but do not allow the user to automatically publish to anydesired format without some manual intervention by the user. Forexample, the Pagemaker software by Adobe Systems, Inc. is one approachto solving the problem of automating manual paste-up. Pagemaker can beused to create elaborate documents, as long as the user is already askilled graphic designer. However, Pagemaker is unable to automaticallyturn text and pictures into sophisticated designs on its own. Likewise,the Adobe Acrobat software addresses the problem of imitating paperdocuments on screen. However, Acrobat still does not allow for automaticredesign or reformatting based upon changing an output media. TheInternet Studio software available from Microsoft Corporation is able totake a document and to separate form from content. However, thissoftware is only capable of producing Web pages. If the user desires totake the same information and turn it into a different media such as adata sheet, a manual or a brochure, the user must manually do all of thereformatting his or herself.

In addition to the inability to automatically adjust content and/ordesign for a desired media, prior art software tools typically onlyprovide a two-way separation between the form and content of a document.In these prior art tools, content structure is typically related to theoutput media, and the content structure is also tagged as to how itshould be presented in the output media. This presents problems when thesame content must be rendered in different output media. In addition,other software tools may be able to render a document into a differentmedia, but are unable to maintain the original design and therelationship between the content elements of that original design.

Also, SGML-based tools and style sheet tools (included with many wordprocessors and desktop publishing tools) may allow the intent of contentstructures to be recorded as tags, but do not allow a specific design tosaved and reused, and automatically rendered to a medium. Also, databasepackages, other forms packages, report packages and "long document"systems (such as Interleaf) have a variety of drawbacks. They do notprovide computation-intensive solutions to a design problem such asglobal fitting, flexible design changes or media changes. They do notsupport rich design structures, nor do they support interactiveexperimentation. Also, they do not allow for repurposing/multipurposingamong different media.

Therefore, it would be desirable for a user to be able to author adocument once, and then have a system to adapt the documentautomatically for changes in the content, design or output media whilemaintaining the relationships between the content elements of thedocument.

SUMMARY OF THE INVENTION

An embodiment of the present invention provides for a three-wayseparation of the information in a document. In a particular embodiment,the document is separated into its content, design and media aspects.Through this representation, an embodiment of the present invention isbetter able to support automatic rendering to multiple forms of media.For example, embodiments of the present invention support the automaticintegration, composition and layout of content from multiple sourcesinto intelligent dynamic document templates instantly publishable inmedia such as print, Intranet, Internet, and in an OLE embedding.

In one embodiment, a content drop table data structure is used to assistin dropping content onto a composition. The drop table is associatedwith a receiving component type in a composition. The drop tableincludes a component type indicator indicating with which type ofcomponent the content drop table is associated. The table has twocolumns of types that may be matched. A first list of potential droppedcontent types is representative of the types of content that may bedropped upon a receiving component type in the composition. A secondlist includes subcomponent types that are compatible with the receivingcomponent type. Thus, for each potential dropped content type of in thefirst list, there is an associated group of at least one subcomponenttype from the second list of subcomponent types that indicate the typeof subcomponent that matches with the potential dropped content type.When a content object having potential dropped content types is droppedupon a component of the composition, a matching subcomponent type may bedetermined by reference to the content drop table.

In another embodiment, a computer-implemented method adds content to acomposition having a chosen design and automatically calculates a layoutfor the composition. The composition is represented by components in adesign tree and includes content elements associated with thecomponents. Initially, a content object is dropped upon a receivingcomponent of the composition. The dropped content object has at leastone potential type. A potential type of the content object is matchedwith a compatible type of the receiving component. Next, the design treeis modified to incorporate the content object, and media layout valuesare calculated for each content element of the composition. The designtree is used in the calculations such that the content elements are laidout in said chosen design.

In various embodiments, the dropped content may change the existingcontent, may replace it completely, or may be inserted into the existingcontent. Also, dropped content may change properties of the existingcontent, or may result in new components being created in the designtree. If content is dropped in an incorrect location, a tree structureof the present invention may be used to determine where to place thecontent.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention, together with further advantages thereof, may best beunderstood by reference to the following description taken inconjunction with the accompanying drawings in which:

FIG. 1 shows diagrammatically how an embodiment of the present inventionmay be used to input content elements, a design description and a mediaspecification in order to produce a new information presentation.

FIG. 1a is first example of a user interface for a travel brochureaccording to one embodiment of the present invention.

FIG. 1b is second example of a user interface for a travel brochureaccording to one embodiment of the present invention.

FIG. 1c is third example of a user interface for a travel brochureaccording to one embodiment of the present invention.

FIG. 1d is fourth example of a user interface for a travel brochureaccording to one embodiment of the present invention.

FIG. 2 illustrates an embodiment of the present invention in which asaved document is input and composed for a particular medium in orderthat it may be rendered in a particular information presentation.

FIG. 3 illustrates various types and functions of content, design andmedia.

FIG. 4 illustrates examples of possible design components in a componentclass derivation tree according to one embodiment of the presentinvention.

FIG. 5a illustrates symbolically the internal structure of a compositionin an embodiment of the present invention in which the composition isrepresented by a content facet tree, a media facet tree and a designfacet tree.

FIG. 5b illustrates in greater detail the composition of FIG. 5a.

FIG. 6 illustrates an example of how structured text may be representedin one embodiment of the present invention by using content, design andmedia facets.

FIG. 6a illustrates another example of how a list item of a memo may berepresented in one embodiment of the present invention by using content,design and media facets in a containment relationship.

FIG. 7 shows in detail the footer component of FIG. 6 having variousproperties that are represented in a content facet, a design facet and amedia facet.

FIG. 8 illustrates an example blank brochure composition rendered forpaper media.

FIG. 9 shows an example brochure having two products that is renderedfor paper media.

FIG. 10 shows the brochure of FIG. 9 to which another product has beenadded.

FIG. 11 shows the brochure of FIG. 9 after the design has been changed.

FIGS. 12a, 12b and 12c show the brochure of FIG. 9 after it has beenrendered to HTML media.

FIG. 13 illustrates an embodiment of the content tree for the brochureof FIG. 9.

FIG. 14 illustrates an embodiment of the design tree for the brochure ofFIG. 9.

FIG. 15 represents an embodiment of the media tree for the brochure ofFIG. 9 rendered for paper media.

FIG. 16 illustrates an embodiment of the media tree for the brochure ofFIG. 9 rendered for HTML media.

FIG. 17 illustrates in more detail a particular component from the HTMLmedia tree of FIG. 16.

FIG. 18 illustrates in more detail an embodiment of the relationshipbetween the content, design and media trees for various of thecomponents of the brochure of FIG. 9.

FIG. 19 illustrates a process by which structured text may be added to acomposition according to one embodiment of the present invention.

FIG. 20 illustrates an embodiment of the composition of FIG. 19 beforethe new structured text has been added.

FIG. 21 illustrates an embodiment of the composition of FIG. 19 afterthe new structured text has been added.

FIG. 22 illustrates an embodiment of a content drop table for aparticular component.

FIG. 23a illustrates an example of a product layout in a particularbrochure.

FIG. 23b is an embodiment of the design tree for the product layout ofFIG. 23a.

FIG. 23c is an embodiment of the media tree for the product layout ofFIG. 23a.

FIG. 23d shows the relationship between the design tree and the mediatree for the product layout of FIG. 23a.

FIG. 24 illustrates an embodiment in which media division objects areassociated with various components of the media tree of FIG. 23c.

FIGS. 25a-25f illustrate examples of possible geometries for a one-waymedia division.

FIGS. 26a-26h illustrate examples of possible geometries for a two-waymedia division.

FIGS. 27a-27e illustrate examples of possible geometries for a three-waymedia division.

FIGS. 28a-28e illustrate examples of possible geometries for an N-Waymedia division.

FIG. 29 is a flowchart illustrating a method suitable for adding newcontent to a composition according to one embodiment of the presentinvention.

FIG. 30 is a flowchart illustrating one embodiment of the match contentstep 512 of FIG. 29.

FIG. 31 is a flowchart illustrating one embodiment of the process matchstep 516 of FIG. 29.

FIG. 32 is a flowchart illustrating one embodiment of the place/replacesteps 616 and 624 of FIG. 31.

FIG. 33 is a flowchart illustrating one embodiment of the fit content tomedia step of FIGS. 29, 37 and 38.

FIG. 34 is a flowchart illustrating one embodiment of the compute mediatree step 708 from FIG. 33.

FIG. 35 is a flowchart illustrating one embodiment of the solve mediadivision step 782 of FIG. 34.

FIG. 36 is a flowchart illustrating one embodiment of the recomputecontent scale factors step 720 of FIG. 33.

FIG. 37 is a flowchart illustrating a method of choosing a new designfor a composition according to one embodiment of the present invention.

FIG. 38 is a flowchart illustrating a method of rendering a compositionto a new medium according to one embodiment of the present invention.

FIG. 39 shows a typical computer system suitable for implementingvarious embodiments of the present invention.

DETAILED DESCRIPTION OF THE INVENTION

The present invention relates to a number of features that facilitateautomatic and dynamic composition and recomposition of program objectsincluding design descriptions, content elements, and output mediaspecifications. Unlike conventional word processing, desktop publishingor Web authoring software, the present invention dramatically improvesdocument production in at least three ways. The present invention isable to automatically integrate a wide variety of types and combinationsof content (e.g., text, pictures, data, graphics, OLE objects, video,sound, etc.). Also, the invention automatically performs documentlayout, styling and fit of specified content elements into a selecteddesign. Additionally, the invention is able to automatically reformatthe content to a new document type or design, to support a wide varietyof distribution mechanisms, including, for example, the printed page, acomputer screen presentation, as an OLE embedded object, or via theInternet as an HTML page. Various embodiments of the present inventionuse a combination of content, design and media features to improvedocument production.

In some embodiments, powerful content processing capabilities makeassembly of content from a wide variety of resources (another disk, overa network, from the Internet, for example) as simple as executing a dragand drop action. These capabilities provides a significant benefit tothe user by addressing two key user problems: (1) the frequentdifficulty of incorporating data from disparate file formats; and (2)the awkward and time consuming content preparation work typicallynecessary in manually preparing multi-format content (e.g., removingprogram specific formatting commands and symbols).

Automatic layout and formatting may be also be performed using built-ingraphic design expertise. Embodiments of the present invention willautomatically fit content to the selected design (newsletter--modern,European, wired, traditional, etc.), represent it in the selected mediaview (printed page, screen, HTML, etc.), position text and graphics,change type specifications, jump stories, add pages, size graphics, ormake other needed adjustments to the layout. If the user changes text orgraphics schemes in the document, or even the overall design, the fit isautomatically recalculated and necessary adjustments are made to formatand/or style. These intelligent design features make it possible todeliver sophisticated design expertise to non-design professionals in"unbreakable documents". These features empower users to make designdecisions based on what is appropriate and "looks good", rather thanbeing constrained by their particular level of design or toolsexpertise.

The concept of "fit" also refers to every aspect of how a document isrendered to a particular medium. For example, various properties ofcontent elements help to express the design of that element or of theoverall document. These properties affect the layout/format of anelement, although may not necessarily affect the overall layout of thewhole document. In other words, even though these properties may notaffect the overall "fit" of a document, the are nonetheless an aspect ofdesign. Examples for text are: font size, typeface, italic, bold,underline, drop capitals, foreground and background colors, andinter-character/line spacing. Examples for images are: crop rectangle,pan position, zoom factor, bit mask and palette. Examples for generalcontent are: borders, orientation, and serial/parallel arrangement ofsubcomponents.

Embodiments of the present invention also support automatic reformattingfor multiple media outputs. A "Write Once--Publish Anywhere" work flowis supported by eliminating the complex tasks of totally reformatting orrecreating a document for publication in an alternative medium (i.e.converting a printed newsletter into a graphically rich Internetdocument.) This media-intelligent automatic reformatting eliminates theneed for a user to own multiple authoring tools for content assemblysuch as PageMaker, Express (from Quark, Inc.), Publisher (from MicrosoftCorporation), Web and multimedia authoring tools, or other DTPapplications. This automatic reformatting eliminates this need formultiple tools by automatically re-composing the output to a widevariety of media environments. Embodiments of the present inventionallow any business user with basic computer skills to be able to quicklyand easily produce professional-looking documents ready for a variety ofmedia such as print, screen, the Internet, etc.

Embodiments of the present invention may be used within a variety ofapplication programs. Turning now to FIG. 1, an embodiment of a use ofthe present invention is shown at 10. An application program 12 thatembodies the present invention has a design engine module 14 and afacets engine module 16. The application program 12 also includes a userinterface 18 which currently is presenting a composition 20. If the userwishes to modify the composition, the user may add new content elements,a new design description or a new media specification through the userinterface or the like in order to be processed by the applicationprogram 12. Once the processing is complete, the application programoutputs the new content, design and media to be presented through userinterface 18'.

In this example, the user has added a new content element 22 that nowappears on the user interface 18' to the right of the content element20'. The user has also specified a new design description for thecontent elements. Content element 20' shown in user interface 18' nowhas a design in which the text appears below the image, instead ofappearing to its right, as in the original content element 20 of userinterface 18. It is also possible for the user to keep the content thesame and modify either the design of the composition or the medium towhich is output. Generally, a user may change one or all of the content,design and media aspects of the composition and have the composition beautomatically adjusted by the application program 12.

The content elements that are added to the composition may be of anytype and may take a wide variety of different forms. By way of example,representative types of content include text, images, data, graphics,OLE objects, video, sound and others. Also, these content elements maybe dropped or located in any suitable manner at any location within theuser interface. The design description specified by the user may also beof any kind. By way of example, the design specified by the user may bea custom design that the user has created through the use of theapplication program 12, or the design may come from a design templatethat is stored in a design catalogue of the application program 12. Manydifferent types of designs are possible. By way of example, the user mayspecify a horizontal brochure format, a vertical brochure format, asingle page or multi-page format, or other designs such as reports,newsletters, memos, home pages, white papers, schedules, programs,agendas, calendars, flyers, tables, catalogues, galleries, Web sites,maps, organizational charts, slides presentations, etc.

Additionally, the user may specify any desired medium to which thecomposition will be rendered. By way of example, a medium typically maybe a sequence of screen pages of a computer, paper pages, a Web site onthe Internet written in Hypertext Markup Language (HTML) or any othersuitable language, a site on an Intranet system, or an OLE embeddedobject. Other types of media are possible as well. For example, thecomposition may be rendered to live HTML (perhaps incorporating JAVAapplets, or Shockwave objects etc.), a multiple page OLE format, amultimedia format, a three-dimensional HTML format (VRML), an audioformat, a TAPI format, a universal in-box format, or any abstract mediumthat may be the subject of further automated processing (such as data tobe output to an Excel model that is not directly consumable by theuser).

The user interface 18' illustrates an example of the application program12 rendering the user specified content, design and media for a computerscreen format. The composition may be rendered to the screen format inany suitable manner such as by standard window drivers and API's knownin the art. The composition may be rendered to paper media by specifyingregions on paper pages including formats and styles where contentelements are to be printed. A composition may be rendered to HTML formatby also specifying regions within Web pages including formats and styleswhere content elements are to be located, and by translating theinformation into the HTML language. The design of the compositioncontains knowledge of what HTML is able to express, thus allowing atranslation into HTML. In other words, the expressible geometriesavailable in an HTML format are included in the design facets of manycomponents of a composition. This information may be inherited fromparticular prototype components of the system. A composition may berendered to an OLE embedded object by using a standard set of publishedinterfaces such as those available from Microsoft Corporation.

The application program 12 that includes the design engine module 14 maybe any suitable application program that is able to take advantage ofthe capabilities of the design engine. Alternatively, the applicationprogram 12 may include an embodiment of the design engine 14, or maycall a server application that includes or calls the design engine 14.By way of example, the application program 12 may perform an automatic,server-side generation of HTML pages or other presentations. This may bedone dynamically without user intervention. And the content used maycome from a database or other data source. In addition, the designengine of the present invention may be used with a wide variety of userinterfaces, such as user interface 18.

The design engine 14 defines a composition framework and framework rulesfor representing documents and their components. The design engine 14also contains a knowledge base of professional document and componentdesigns commissioned specifically for this purpose. In one embodiment,the design engine models documents as deeply nested combinations ofthree dimensions: content elements, design descriptions and mediaspecifications. The design engine manages these dimensionsindependently, resulting in more efficient, richer, more flexible andmore automated interaction for a user creating documents. The designengine automatically adjusts the composition whenever the user changesany of these dimensions. With this capability, the design engine may usethe same interface to support compositions that vary from printedmaterials and screen presentations to other media such as the World WideWeb.

The design engine takes as input content elements, a design descriptionand a media specification, and is renders the content to the medium inaccordance with the design description. The design engine is constructedsuch that it inherently has the capability to present the information inany one of a variety of media. Preferably, the design engine is builtupon technology provided by the facets engine as described in U.S.patent application entitled "Method and System for Implementing SoftwareObjects", Ser. No. 08/546,316 filed Oct. 20, 1995, by inventors Marc B.McDonald and Michael B. Orr, which is hereby incorporated by referencein its entirety. The facets engine provides an object-oriented platformupon which the design engine may be built. A brief explanation of thefacets technology will now be provided below to assist the reader inunderstanding particular embodiments of the present invention. However,it should be appreciated that in other embodiments, the capabilities ofthe design engine may be implemented using a technology other than thatprovided by the facets engine.

FIGS. 1a, 1b, 1c, and 1d show examples of a user interface 18 as itmight appear in conjunction with a "China Tours" brochure. FIG. 1a showsa user interface 25 having a list of available content on its left-handside, along with a one page "China Tours" brochure having twophotographs/destinations and accompanying text. FIG. 1b shows a similaruser interface 26 in which a third photograph/destination andaccompanying text (currently blank) has been added to the bottom of theone page brochure. The layout has been adjusted to accommodate all threephotographs and text on one page. FIG. 1c shows a similar user interface27 showing in detail a different design for the "China Tours" brochurein which a small logo (from the top of the brochure of FIG. 1a) nowappears next to each photograph in the brochure. In this figure, eachphotograph now appears on its own page, such as might be presented in aWeb page format. FIG. 1d shows a variation of a user interface 28 havingavailable content on the left-hand side, and also a listing on theright-hand side of the hierarchy of the design for the brochure, and theassociated content for each design component.

FACETS TECHNOLOGY

As described in the above-referenced application "Method and System forImplementing Software Objects", the use of facets eliminates significantproblems associated with multiple inheritance in object-orientedsystems, such as brittle classes and derivation conflicts. These issueshave prevented object-oriented programming from fulfilling the promisedrevolution of software productivity and code reuse. By using facets,however, programmers can easily create objects which do not depend oninheritance in native class libraries, yet still have access to all ofthe properties and behaviors of the native class objects. In otherwords, facets eliminate significant problems associated with inheritancein object-oriented systems.

Increased programmer efficiency and easier implementation of complexmodels are achieved through the use of facets by improved encapsulationand inheritance methods that permit the programmer to understand theproperties and behavior of an object without requiring a completeunderstanding of the class library in which the object is based.Additionally, the use of facets avoids inheritance anomalies byspecifying inheritance in accordance with a technique that permits aprogrammer to define an object without resorting to complex arbitrationrules on inheritance hierarchies. Complex applications can be designed,compiled and understood by a programmer with the full benefits ofobject-oriented coding, including reusable code and single and multipleinheritance.

Specifically, the use of facets allows different properties of an objectto be declaratively inherited from different parent objects. A set ofproperties within an object can be grouped together and treated as aspecific aspect of the object. Such an aspect is referred to as afacet-type property subgroup of the object (or more simply as a "facet"of the object), and a facet can be inherited from different parentobjects. Inheritance of a property within a facet may be based uponeither an "Is-a" or a "Has-a" relationship. Property values inheritedbased upon an "Is-a" relationship are independent of the location of theobject, and the parent of each facet is determined by declaration.Property values inherited based upon a "Has-a" relationship will derivefrom the parent object or objects in which the child object is placed.

An object may be placed in more than one container parent, withproperties within each of the facets of the object being inherited froma different one of its parent containers. The parent from which eachfacet is inherited can be specified. Alternatively, when a child objectis contained in multiple parents, each child object facet is inheritedfrom the parent in which the child object is contained in a parentobject facet having the same name as the child object facet. Individualproperties within a facet can be declared as inherited or not inherited.The values of properties that are not inherited will be independent ofthe property values in the parent object. Thus, the properties that achild object inherits is controlled by the child object itself: by itsplace in a container object and by declaration in the child of whichproperties are inherited from the parent.

By specifying inheritance at the level of a facet and individualproperties, a programmer can define an object to behave the way he orshe wants; the properties and behavior of an object will not depend onarbitration rules operating on complex multiple inheritance hierarchies.A programmer can understand the properties and behavior of an object bylooking at its declaration without having to understand the completeclass library upon which it is based. Thus, the use of facets technologyis preferred for one embodiment of the design engine, but is notrequired.

DOCUMENT COMPONENT FRAMEWORK

Referring next to FIG. 2 the relationship between an application program12, a document 32, a composition 38 and its information presentation 36in accordance with one embodiment of the invention will be described inmore detail. The document 32 represents information that has been savedby the user within a computer system to persistent storage. Thisdocument may include any combination of content elements, designdescriptions and media specifications. Once the application program 12is executing, the document 32 may be loaded into the application programas an open document 32' and may appear to the user through a userinterface 18. Once the document 32' has been composed for a particularmedium, it appears as a composition 38 having a particular layout 39.That is, composition 38 represents the document 32' in a particularlayout/format (or style).

Once the application program 12 is directed by a user to render thecomposition (via a "publish" command or the like), the composition 38 isrendered to a particular medium in the form of an informationpresentation 36. This information presentation may appear initially in aform suitable for viewing on a computer screen. For example, if thepresentation has been rendered for paper media, the user interface mayshow this paper media in a print preview format. Likewise, if thepresentation has been rendered for a Web site, the user interface mayshow these Web pages on the computer screen for the user to view. Theinformation presentation 36 represents the actual pages created by theapplication program for paper, HTML or any other medium.

One definition of terms suitable for understanding the present inventionfollows. The document component framework creates value ultimately bysupporting the construction of information presentations. An informationpresentation is typically a materialization of information in a way thatmakes it accessible to a human being or an automated process. Apresentation may be physical (e.g., a set of printed pages), ephemeral(e.g., an on-screen image), distributed (e.g., a hypermedia Web),abstract (e.g., a source code file), or in some other form. Also, apresentation may be static (e.g., printed pages) or dynamic (e.g., astock ticker). In one embodiment of the present invention, apresentation may be a sequenced set of printed pages, a sequenced set ofscreen images able to be viewed from within a user interface, an OLEobject embedding, or a single-rooted set of linked HTML pages.

A composition typically combines content, design and media in a usableform. In general, content is information substance, in forms such ascommunication-bearing text, images, symbols, data structures and other.Media, in general, is presentation space, in forms such as paper,sequenced computer screens, linked hypertext pages and other. A designis typically information presentation specifications, in forms such asdefinitions of allowable content and media combinations, and methods forrendering presentations of particular content to particular media. Thus,a composition may be viewed as a combination of compatible content,media, and design objects capable of producing a specific presentationof specific information in a specific form. In one embodiment of thepresent invention, design, media, and content are the primary facets ofa composition. A composition typically results from a request to rendera media-specific presentation of an open document. The present inventionis able to elaborate the document structure as necessary to computefully explicit layout and drawing information.

A composition may be viewed in different ways. In one aspect, thepersistent, storable form of a composition is referred to as a document.A document may be stored in many forms. By way of example, a documentmay be a computer file (or other long-lived object) designed to serve asthe basis for a composition. The concepts of a composition and adocument are closely related. However, while a composition--a specificconfiguration from which a presentation is rendered--may fully specifytarget media and an explicit layout, a document is likely to leave theseproperties unresolved so as to support different user choices atdifferent times. A document object may support presentations to a widevariety of supported media; this generally corresponds to the user'snotion of an identifiable publication. The fully resolved nature of acomposition, on the other hand, makes it a more convenient referencepoint for understanding the structure of the component framework. Inother words, a document is typically a persistent object that maysupport multiple media, while a composition typically represents asingle media presentation.

CONTENT, MEDIA, DESIGN FACETS

As suggested in the preceding sections, in one embodiment of the presentinvention the document component framework's modeling strategy involvesfactoring a composition into independent content, media, and designfacets. This approach has many advantages: it is simple to grasp andpromises powerful capabilities, such as one-step experimentation withdesign changes, easy repurposing of content for different media, and theability to save design representations that are far more adaptable thantraditional templates. Each of these facets may have its ownhierarchical structure, and the hierarchies may not necessarilycoincide.

Thus, because information may be viewed in many instances as content,media and design, there are various advantages to representing acomposition by a content facet, a media facet and a design facet. Theseadvantages are further discussed below, along with a more detailedexplanation of the content, media and design facets.

The content facet of a composition may be considered to be thecommunication substance it contains in forms such as text, images, andsymbols that may, at least in principle, be broken away from otherfactors as pure information. In a passage of body text, the wordsgenerally function as content, but the typeface generally does not.Particularly in the case of text (and other elements "attached" totext), content has an inherent structure that interacts with, but is notdetermined by, the structure of media (presentation space) or design. Ina magazine article, for example, title, headings, body passages,illustrations and other elements form a hierarchical structure that maycollide unpredictably with other structures such as page and columnbreaks. Thus, it makes sense to separate this pure content from theartificial constrictions of a page break in paper media, for example.

Additionally, in any document, content and other presentation featuresmust be coordinated across a hierarchy of scales from individual textcharacters and image pixels up to the entire document, or even a seriesor family of documents. At all levels of scale, recognizable featuresexist that have some independence from a specific document or documenttype. One example might be an array of illustrations containing titles,images, and captions in a particular relationship. Such a structurecould be reused across catalogs, reports, newsletters, and otherdocuments keeping the same relationship, but the specific formatting ofeach element would likely change according to context. In this example,an entire hierarchy of features and relationships is preserved from onemedium to another, but its presentation is interpreted differently indifferent settings. This is characteristic of situations involving thereuse of document elements. Thus, separation of content from mediaallows reuse of the content in different media while preservingrelationships.

The media facet of a composition is structured hierarchically by theallocation of presentation space (or regions) to the different features(or content elements) of a composition. This allows layout and localityconcerns to be separated from design and rendering concerns, andrecognizes the fact, noted above, that the hierarchical structure ofpresentation space is distinct from the structure of content. In thisinterpretation, subordinate parts of a composition are associated withtheir own media objects, which are combined, perhaps in multiple levelsof aggregation, to form the composition-level media facet. In thesimplest cases, this reduces to a facet-oriented terminology for layoutin which each page (presentation space) is divided into regions whichare assigned to the various parts of a composition.

In any chosen presentation space, any assemblage of content may berendered in an unlimited variety of ways, but a particular compositionis rendered in a specific way. Everything about a composition thatdefines its specific renderings, among all those possible for itscontent, is considered to make up that composition's design facet. Thus,the framework interprets design not only as "style", in the sense ofrendering choices made "at the last minute" in a superficial way, but asthe controlling factor for the whole logical construction of a documentand the compositions it supports.

Like content and media, a composition's design may exhibit ahierarchical structure, with recognizable parts ranging from theprimitive presentation styles of text and graphic content elements, upthrough the organizational and presentation modes of intermediatestructures, to the overall structure and appearance of the entirecomposition. At all but the lowest level of this hierarchy, design isexpressed by assembling components of a more primitive nature and makingassociations with content and media. The use of components will now bediscussed in more detail.

COMPONENT ARCHITECTURE

A variety of object-oriented programming techniques may be used toimplement the present invention. By way of example, techniques such asthe class-instance model, the object programming model and the objectprototype model may be used. In a preferred embodiment, the notion ofspecialization in the component framework is based upon object-prototyperelationships, rather than class-instance relationships. Also, inaddition to derivation-based inheritance, in which an object inheritsproperty bindings from the prototype object on which it is based, thecomponent framework relies upon containment-based inheritance asdeveloped in systems such as Apple Computer's Hypercard and Asymetrix'sToolBook. In containment-based inheritance, an object inherits propertybindings from a container or owner object.

The fundamental abstraction supporting the framework is the component.In the problem domain view a component is a coherent entity within adocument or composition that can be independently created and destroyed,inserted and removed, and/or changed in state; the entire document orcomposition is considered to be such a component. And in the softwarearchitecture view a component is a software object representing such anentity as a node within a composition's content, media, and designhierarchies.

The most typical component is a straightforward document part such as aparagraph, list, illustration, or article. In order to serve as thecommon prototype for all framework objects, however, the componentconcept is generalized to cover encapsulated content objects (trivialmedia and design facets), encapsulated media (trivial content anddesign), document and component "blanks" on a palette (place holdercontent), and entire documents and compositions (no container in thedesign hierarchy). The component prototype thus provides the commonrepresentation that brings entire documents and compositions,intermediate-level objects, primitive content elements, and media into asingle, interconnected structure.

Components may reside simultaneously in several independent hierarchiesand therefore in several different container objects. For instance, thedescriptive text body in a catalog item resides simultaneously within atext file (content), within one or more regions of one or more pages(media), and within a catalog item (design). The three facet-specificcontainment contexts, together with derivation from a prototype object,define a total of four object relationships that help determine acomponent's property bindings and thus its state as a software object.

This view of a component implies that the attributes of a composition'scontent, media, and design facets are distributed across the states ofcomponent objects. The architecture of the framework also preserves theconverse property: the attributes of a component are partitioned intocomponent-level content, media, and design facets that reside within themore encompassing facets of the entire composition. A more detailedillustration of the different facets of a component will be discussedbelow with reference to FIGS. 5a, 5b, 6 and 7. Thus, the various facetsof components not only exhibit different structures (such as a contenttree, a media tree and a design tree), but they also can be representedusing independent sets of properties (state variables).

An example of the many possible types of design components is shown inFIG. 4. FIG. 4 at 130 shows a component class derivation tree thatincludes many of the common types of design components that may be usedin a design by the user or that may be used in a standard design in thedesign catalogue of the system. For example, beneath the base component"tabular", is the leaf component "table". A user may include thiscomponent "table" within a design if the user wishes to place a table ata particular point in the document. Typically, a user uses the leafcomponents to arrange a particular design; alternatively, a standarddesign from the system catalogue will use these leaf components to buildup a particular design. When creating a custom design, the user mayselect from these components through the user interface in order tocreate the new design. Additionally, the user may select a standarddesign including components that are pre-arranged from a system designcatalogue.

Included with each component is knowledge about how that component maybe rendered in a particular media. For example, for rendering to HTML,macros may be used for the HTML elements in rendering to a particulargeometry. A further discussion is included below with reference to FIGS.12a-12c.

Returning now to a discussion of components in general, other types ofprototype components will now be described. Derived from the componentprototype are three other kinds of prototypes that support the separatedrepresentation of content, media, and design. These other prototypes arecontent objects, media objects and design objects and are all types ofcomponents.

Content objects combine content encapsulations with unbound media facetsand unspecified design facets. Content can be encapsulated either as alink to an external object, or as an embedding. Built-in contentencapsulations represent both free-standing objects, such as text files,and nested sub objects, such as the sections and paragraphs of textfiles. Because a content object is a component, though a lopsided one interms of facets, it can also participate in containment inheritance bybeing specified as the content facet container (parent) for anapplication-level component.

Media objects combine media object encapsulations with unbound contentfacets and unspecified design facets. Built-in media encapsulationsrepresent both free-standing objects, such as printed documents, andnested sub objects, such as the individual page regions associated withcomponents of printed documents. Because a media object is a component,though a lopsided one in terms of facets, it can also participate incontainment inheritance by being specified as the media facet container(parent) for an application-level component.

Design objects implement specific styles for presenting compatiblecontent to supported media. A design is considered to be a documentshell, or just shell, if its instantiations are thought of primarily asfree-standing documents that do not require a container context. This isa soft distinction, since most document shells may also be constructedinside a containing document. A design is considered to be a componentformat, or just format, if its instantiations are thought of primarilyas subcomponents of larger documents. Sometimes this is a softdistinction, since some component formats may also be constructed asfree-standing documents.

In one embodiment of the content, media and design triad of facets,design has a separate but somewhat more than equal status, at least fromthe point of view of a design-centric framework application. This isreflected in one possible interpretation of design facet containment asthe logical component structure of a composition, and in a preferredtechnique of deriving application component instances from designprototypes. Thus, in this embodiment, a component that has non-trivialbindings for all three facets is a specialization of some built-indesign type, as well as typically being contained in another componentvia the design facet (logical structure). In the content and mediafacets, only the containment relationship is available. This accountsfor the extra indirection needed to reach "specific" content and mediaobjects from document components--an extra level of containment is beingused to provide a derivation path for the content and mediarepresentations. This can be though of as asserting that the componentinstance is contained in its "own" content and media facets.

An example 42 illustrating this containment is shown in FIG. 6a. This isan example of a memorandum that has only a list with a number of listitems. The component "Specific List Item" represents one item from thatlist, and is contained in each of a content, media, and designhierarchy. Also, this "Specific List Item" component is a specializationof a Generic Component Prototype. The content facet of "Specific ListItem" shows the text "aaa" of the item is contained within Body Text,which in turn is contained in a Text File. The media facet of "SpecificList Item" shows the specific item is contained within a particularRegion (having a Size), that in turn is contained in a Page, which iscontained in a Printed Page Set. The design facet of "Specific ListItem" shows how the specific item is contained within a List, which inturn is contained in a Memo.

CONTENT, DESIGN, MEDIA TREE EXAMPLES

As discussed above, because information may be viewed by content, designand media aspects, it is helpful to represent this information in acontent tree, a design tree and a media tree within a composition asshown in FIGS. 5a and 5b. FIGS. 5a and 5b diagrammatically illustratethe relationship between a composition, the content, design and mediatrees, and components. FIG. 6 below provides a particular example of arepresentation of text using components having a content, design andmedia facet.

FIG. 5a shows in more detail a composition 38 from FIG. 2. Thecomposition 38 may be stored as a saved document 32 in a computersystem, or the composition may be rendered to a particular medium in theform of an information presentation 36. The information contained withinthe composition 38 may be divided up in a variety of manners.Preferably, the composition is represented in the form of a content tree52, a design tree 54, and a media tree 56. These trees are preferablylinked in some fashion 57 as will be explained in greater detail below.These content, design and media trees may be based upon any suitableobject-oriented technology as discussed above, although the use of thefacet technology as described above is preferable.

This composition 38 of FIG. 5a is shown in more detail in FIG. 5b. FIG.5b illustrates how, within a composition 38, the content, design andmedia facet trees may be represented in a general sense. In oneembodiment, a composition root object 60 serves as the root for each ofthe content facet tree 62, the design fact tree 64, and the media facettree 66. Each of these trees has an independent but possibly overlappinghierarchy of child components, such as component 70, within anintermediate set of components 68. That is, these three trees may followdifferent paths through the set of components 68. In other words, anindividual component 70 may be unique to only one of the three trees,may be shared between two of the trees, or may be common to all of thetrees. Leaf nodes such as leaf 72, are typically common to each of thethree trees and contain such information as actual content elements andcoordinates for the layout of the content in a particular media. In apreferred embodiment, the composition root object and the leaves arecommon to all three trees. In other embodiments, most or all componentsmay appear in each of the three trees.

Each element of the composition such as an image pixel, a character, agraphic symbol, a word, and up to and including the whole document, maybe associated with a component. Typically, a component of largegranularity is created by a user action, such as a user dropping aparagraph of text onto a composition. This paragraph would berepresented at first by a paragraph component. This paragraph componentmay be fragmented as needed in order to fit content to a layout. Forexample, each line of the paragraph may be represented as asubcomponent. The fragmentation process is also quite dynamic. Forexample, fitting a paragraph involves trying and discarding manypossible fragmentations, and this fitting may be redone after eachsignificant user action. Editing may also create new subcomponents bycreating attribute changes or by forcing a refit. For example, if amiddle portion of a line of text is selected to appear in italics, thenthe line of text would be fragmented into three subcomponents. The newlyitalicized portion of the line would be one subcomponent, while theportions before and after it would be the other two components.

In one embodiment of the present invention, each component has a contentfacet, a design facet, and a media facet. Although, a facet for aparticular component may be unused, as in the case of a media facet treecomponent that is located between two nodes of the design facet tree butnot actually in the design facet tree. In this example, the design facetof this media component may be unused because there is no link to thedesign facet tree.

FIG. 6 illustrates how a piece of structured text 80 may be representedusing an embodiment of the content, design, and media facet trees of thepresent invention. The structured text shown at 80 has a title "WHO WASKILROY?", a first heading "Here at One Time", and a second heading"Missing in Action". The first heading also has a body of text A and thesecond heading has a body of text B.

This structured text 80 may be represented using the content and designfacets of components as follows. In this example, the root object forthis structured text is the Structured Text component 84. This component84 may also have a link (not shown) to a container Document. The designtree representation for this structured text 80 may be determined byfollowing the design facet links from Structured Text component 84 ondown the hierarchy of design facet links. For example, Structured Textcomponent 84 contains (by way of a design facet link) Title component86. Title component 86, however, contains both a Heading One component90 and a Heading Two component 92 by way of their respective designfacet links. In turn, Heading One component 90 contains Body Text Acomponent 94, and Heading Two component 92 contains Body Text Bcomponent 96. Thus, it can be seen how the design tree for a particularcomposition is represented by links formed through the design facet ofthe various components in the composition.

Likewise, the content tree representation of the structured text 80 maybe determined by following content facet links from each component. TheTitle component 86 has a content facet link to the title of thestructured text 80, namely "WHO WAS KILROY?". The component Heading One90 has a content facet link to the first heading "Here at One Time".Similarly, each of the other headings, and body text has a content facetlink pointing to the particular content represented by that component.The link from a component to another component or from a component tocontent may be formed in any suitable fashion. Which inheritance linksare used in a given tree structure is determined by part of the objectdefinition for each component on a per-property basis. These propertiesof a component are divided up and inherited via different paths. Asdescribed above, the concept of facets introduces the notion by whichthe properties of a component are partitioned such that the differentfacets of that component participate in the three different containmenthierarchies, namely the content facet tree, the design facet tree andthe media facet tree. Using the concepts of facets is preferred becauseof improved correspondence with document reality.

Although not shown in FIG. 6, these components are also linked via theirmedia facets. The media facet of a component, such as Title 86, containsproperties including the positioning of the title and its dimensions. Itis possible that there may be other components within a composition thatparticipate only in the media facet tree via their media facets, and arenot part of either the content nor design facet trees. Likewise, theremay be components that participate in each of the content, design andmedia trees through their respective content, design and media facets.Further examples showing a media facet tree for a composition aredescribed below.

FIG. 7 is an example of a footer component 120 showing in detail itscontent, design and media facets. The footer component 120 may have anynumber of properties defined upon it or that it has inherited through ahierarchy of components. By way of example, selected propertiesassociated with the content facet, design facet and media facet areshown for this footer component. It should be appreciated that manyother types of properties may be associated with each of these facets,and other properties may be defined upon the footer component that arenot associated with any of the facets.

In this example of FIG. 7, the content facet contains a property namedText which has as a value a pointer to a text string containing the textfor the footer. The design facet for this footer component includes theproperties Text Face, Text Attribute, Justification and Relative Size.Example values corresponding to these properties are shown in FIG. 7.The media facet for this component includes the properties X Position, YPosition, Width and Height, which help to define the geographical regionon a page where this footer may appear. These geographical propertiesmay in general be relative to media facet containers. For example,coordinates may be relative to a particular containing region. That is,which particular page a region appears on may be determined by searchingup the media facet links until a Page component is encountered.

It should be appreciated that many other different kinds of propertiesmay be present within a media facet to help define the extent of thecomponent and to locate the component within the content rendering spacefor a particular medium. By way of example, properties such as color,texture, and percent transparency of the media surface may be associatedwith a media facet. A discussion of the extent of a component isdetailed below with reference to FIG. 34 at step 758.

BROCHURE COMPOSITION EXAMPLE

FIGS. 8 through 12c below provide an example of a brochure that may becreated and rearranged in various embodiments of the present invention.Examples of possible content, design and media trees to represent thisbrochure composition follow in FIGS. 13 through 18. FIGS. 19 through 28eprovide further examples of uses of the present invention and itsfeatures, while FIGS. 29 through 38 describe procedures for implementingembodiments of the present invention.

An example of a blank composition that has been created by loading astandard design from a system design catalogue is shown in FIG. 8. Ablank (or empty) composition is one for which a design exists, but nocontent has been added. This blank composition 140 includes a nameplate142 and a title 144. Also included are images 146 and 150 each withcorresponding text areas 148 and 152. A footer 154 is located at thebottom of the blank composition 140. These elements of the blankcomposition are represented in the design tree for the composition bycorresponding components for which no content has as of yet beenassociated. At this point in the process of creating a composition, theuser may wish to drag and drop content onto various portions of thescreen in order to create a more elaborate composition. Alternatively,the user may rearrange the design shown in order to create a customdesign.

An example of a design that has been filled in by the user is shown inFIG. 9. This document design specifies a paper page media fixed at onepage in length. A user may have added content to this design by draggingand dropping text and images from various sources. Alternatively, a usermay also perform editing of the content directly in the compositionthrough the user interface. FIG. 9 shows a brochure composition 160 thatmight be used in the mail order business for fishing supplies. Thebrochure composition 160 has a title "Bass Products, Inc." 162, a fishlogo 164, two products and a footer. The first product has a picture ofa rod 166, a caption "Rod" 168, and product text 170. The second producthas a picture of a net 172, a caption "Nets" 174, and product text 176.A footer "Call In Your Order Today" 178 is located at the bottom of thecomposition. In this example, this brochure composition has beenrendered for a single page of paper media.

FIG. 10 shows the result after the user has added new content in theform of a new product to the brochure composition 160'. In this example,the user has dragged and dropped a new product towards the bottom of thebrochure composition of FIG. 9 and a method of the present invention hasautomatically reformatted and placed the new product along with theexisting products onto the single page of paper media. As shown in FIG.10, a new picture of a hook 180, a caption "Hooks" 182, along withproduct text 184 have been added to this composition.

FIG. 11 illustrates an additional example in which the user has notadded new content but instead has changed the design of the brochurecomposition. In this example, the user has selected a verticallyoriented design in order to produce brochure composition 160". Here, amethod of the present invention has automatically oriented the picture,caption and product text for each product in a vertical order on thesingle page of paper media. For example, the caption "Rods" 168 nowappears below the picture 166, and below the caption now appears theproduct text 170. Likewise, the second product has been orientedvertically. A method of the present invention has automaticallyrecomputed a layout for this brochure based upon the chosen verticaldesign for a paper page medium.

FIGS. 12a, 12b and 12c illustrate at 190 the brochure composition ofFIG. 9 that has been rendered to HTML media. In this example, the singlepage paper media layout of FIG. 9 has been automatically transformedinto a three-page HTML layout suitable for presentation on the WorldWide Web, for example. This rendering to a different media may occurwhen a user executes a command from the user interface to publish thecomposition to a new media.

FIG. 12a shows a title page 192, FIG. 12b shows a first page 194 for thefirst product, and FIG. 12c shows a second page 196 for the secondproduct. The title page 192 of FIG. 12a includes the title "BassProducts" 162 and the fish logo 164 from FIG. 9, but these contentelements have been automatically scaled and located in a differentposition on the title page. Also included in the title page is now a"What's New" link 198 that links the user to another Web page. Alsoautomatically created is a brochure table of contents 198 that includesforward links to "Rod Products" 200 and "Net Products" 202. Alsoincluded on this title page 192 is a link to a "Talk to Us" Web page204. The first page 194 includes the image caption and product text fromthe original FIG. 9. Also included with this first page is the title andfish logo from FIG. 9 and a new link "Home Page" 206 with a backwardlink to the title page 192. The second page 196 for the second productlikewise includes the original image, caption and product text from FIG.9 and also includes the title, fish logo and new "Home Page" link 208.

By various methods of the present invention the content and design ofthe brochure composition of FIG. 9 has been automatically rendered forHTML media. Each component of the composition of FIG. 9 has associatedproperties that dictate how the component will appear in various media.For example, general properties may be used to indicate how a componentshould appear. These properties may include: complex vs. simple, classicvs. modem vs. postmodem, energetic/dynamic vs. calm, warm vs. cool, darkvs. light, elegant/rich vs. plain, fast vs. slow, etc. It should beappreciated that other properties (such as those discussed above) mayalso be used.

Also, by various design choices built into the present invention, eachmedium to which a composition may be rendered contains knowledge abouthow it will appear. Prototypes of objects also contain knowledge. Forexample, specifically programmed into prototypes of media objects aredefaults for primitive content types. In prototypes for content objectsare abstract attributes (such as "emphatic", "playful", etc.) that canaffect the primitive rendering routines for a particular medium. Thatis, the content types set up attributes that can be turned on or off bycomponents and read by a medium. Also, component prototypes setattributes that establish (by inheritance) rendering defaults. Morespecialized prototypes and specific instances of components selectivelyoverride inherited attributes as appropriate. Overrides at the specificinstance level can also be specified interactively by the user. Forexample, via a property sheet. Overrides can be directed to theprimitive media dependent parameters as well. An example of a generalprototype might be "text", while an example of a specialized prototypewould be "title". An example of a specific instance would be aparticular title in a particular document. As is known in the art, a"prototype" refers to a build-in object relatively near the root of the(an) object derivation tree. A prototype is used to transmit pre-builtbehaviors to user-specialized objects while still allowing thepossibility of user overrides.

For example, various new elements have been added automatically to thenew HTML media pages such as the "What's New" link on the title page 192and the "Home Page" links 206 and 208 of pages 194 and 196.

Alternatively, the brochure composition of FIG. 9 may be rendered to anyother suitable media. By way of example, the brochure FIG. 9 may berendered to screen media which is suitable for viewing on the screen ofa computer in order to make slides for a later presentation. A screenmedia format may be similar to the HTML pages of FIGS. 12a, 12b and 12c.Also, it is contemplated that a user may wish to change more than oneaspect of a composition at a given time. For example, a user may wish toadd content to a composition and also to render it to a different media.

Turning now to FIGS. 13 through 18, an embodiment of the brochurecomposition shown in FIGS. 9 through 12c is shown as being representedin embodiments of a content tree, a design tree and a media tree. Itshould be appreciated that the tree structures of FIGS. 13 through 18are illustrative examples of one way of linking the content, design andmedia of the brochure composition. These trees are symbolic ways ofviewing the content, design and media facets of the composition and assuch may appear in other forms while still representing the composition.Showing these trees as separate entities assists in understanding howthe use of facets technology allows separation of the content, designand media of a composition.

In other words, the embodiments of the trees as shown in FIGS. 13through 18 are the internal representation of the brochure composition.As explained above, the content, design and media trees are notnecessarily completely distinct trees, but may share intermediate nodes.Also, the content, design and media trees may typically share a commonroot and may have leaves in common, as will be shown below in theexample of FIG. 18. The formation of these components into hierarchicaltrees may be performed in any suitable manner as will be appreciated byone of skill in the art of object-oriented programming. Preferably, thecontent, design and media trees represent containment relationshipsbased upon an object prototype model that makes use of the facettechnology.

FIG. 13 shows one possible representation 220 of a content tree of thebrochure composition of FIG. 9. The content tree has a root, DocumentContent, and a first node "Bass Products" that represents the title ofthe brochure. Below the title are nodes representing the fish logo, thefirst product (a rod), and the second product (a net). Below the rodimage are nodes representing the caption "Rods" and the product text forthat product. Below the net image are nodes representing the caption"Nets", and the product text for that product. It is contemplated thatthis content information may be organized in a different hierarchicalfashion yet still represent the same information. By way of example, thecaption for each product may be an intermediate node having the image ofthe product and the product text below it as leaf nodes, instead of thecaption being a leaf node. Alternatively, an intermediate node below"Bass Products" may be labeled "Product One" and would have three childleaf nodes, namely the rod image, the caption and the product text. Inone embodiment of the present invention, each node containing content isassociated with a particular component in the design tree. For example,a component labeled "Picture" in the design tree would have a link tothe image of the rod. Such links from design components to contentelements are also shown in FIG. 6.

Thus, the content tree represents the information of the brochurecomposition, apart from any interpretation as a document. However, thecontent tree may change depending upon the media to which it isrendered. For example, when a composition is rendered to HTML format,elements such as the "What's New" or "Talk to Us" links may be added tothe title page as shown in FIG. 12a, thus adding new content elements tothe content tree.

FIG. 14 shows an embodiment of a design tree 230 for the brochurecomposition of FIG. 9 that has been rendered for paper media. The nodeswithin the design tree represent components, such as may be found inFIG. 4. The leaf nodes of the design tree represent the content of thebrochure composition and are shared with the leaf nodes of the contenttree of FIG. 13. In this fashion, it may be seen how the content anddesign trees have different intermediate nodes, but may share the sameleaf nodes, namely the content information. The leaf nodes also containdesign and media information. For example, printing a line of textrequires not only the text (content), but also the font and styleinformation (small scale design), and a region on a particular page(media). Thus, a leaf may contain information suitable for directrendering by an API, driver call, HTML output routine, etc, in order torender the content into a particular information presentation.

The design tree 230 has a root, Document Design, having three componentchildren: Nameplate, Product List, and Footer. The component Nameplatehas subcomponents Title and Logo that are each linked to theircorresponding content. The Footer component has a link to itscorresponding content, namely the string "Call in your order today." Thecomponent Product List has subcomponents Product 0 and Product 1. Thecomponent Product 0 has two subcomponents: Product Illustration andProduct Text. The component Product Text is linked to its correspondingtext. The component Product Illustration has two subcomponents: Pictureand Caption, each of which is associated with its corresponding content.The hierarchical structure for Product 1 is similar to that for Product0 as can be seen in FIG. 14.

From this design tree, it should be appreciated that the components ofthe design are represented independently of a medium to which thecontent may be rendered. That is, components represent elements of thecomposition without having to specify exactly where that element willappear on the paper page. However, it is possible that the design treemay vary depending upon the media to which the content is rendered,although the design tree is still independent of the exact location ofthe content in that media. For example, if the brochure composition wereto be rendered in HTML media, two new components may appear below eachof the components Product 0 and Product 1. For example, componentProduct 0 would have additional subcomponents Product Forward Link andProduct Backward Link. The Product Forward Link would contain contentlabeled "Rod Products", that would provide a forward link from the titlepage to the page of Product 0 (as seen at 200 in FIG. 12a). Likewise,the component Product Backward Link would be associated with the text"Home Page" and would form a link from the page upon which Product 0appears back to the title page of the HTML media (as seen at 206 in FIG.12b).

Also, in HTML media three additional Title components may be created foreach of the new pages. In one embodiment of the present invention, new"Bass Products" components are created for each page of the new media.An illustration of a "Bass Products" component for a particular media isshown below in FIG. 17. The design tree may also change if content doesnot fit within a particular medium. For example, if a large textparagraph does not fit at the bottom of the page, the paragraph is splitinto two pieces. Instead of one component representing the largeparagraph, a new design component is created to represent the secondhalf of the paragraph on the second page. This new component would alsobe added to the design tree. Other component additions to the designtree are possible for other types of media and content rearrangement.

FIG. 15 represents an embodiment of a media tree 240 for the brochurecomposition of FIG. 9 in which the brochure has been rendered to papermedia. The purpose of the media tree is to provide a layout, or locationfor each item of content of the brochure composition. The media tree hasa root, Paper Page Set, with one child component Page 0. In thisexample, the brochure composition is being rendered to a single paperpage, hence there is only one child component, Page 0. Directly belowthe Page 0 component is the content for this brochure composition,namely the title "Bass Products", a fish logo, a picture of a fishingrod, etc. The media tree shares these leaf components with the contenttree of FIG. 13. Also shown in FIG. 15 are the media facet coordinatesfor each item of content indicating a layout position for that contenton the single paper page.

In this simple example, the layout coordinates form the extent of theitem of content. In general, the extent of an item of content indicateshow much of the content rendering space in a particular medium thecontent occupies. By way of example, the extent of a content item may beits rectangular coordinates, may indicate a three-dimensional position,a length of time, a segment of sound, a segment of video or otherportion of a particular medium. In this example, the extent for eachitem of content are X and Y coordinates and the item's width and height.For example, the title "Bass Products", has an X coordinate of 10, a Ycoordinate of 10, a width (dx) of 50, and a height (dy) of 5.Coordinates for the other content items are provided in a similarfashion as shown below each content item in FIG. 15.

In one embodiment of the present invention these coordinates arerepresented as properties of the media facet for the componentrepresenting that content item. It should be appreciated that othercoordinate systems aside from a rectangular coordinate system may beused, such as polar coordinates, etc., in addition to specificationsthat are not coordinates. Thus, it can be seen how the media facet treerepresents the layout for the content of the brochure composition, yetis distinct from the design for that composition.

FIG. 16 shows an embodiment of a media tree 250 for the brochurecomposition of FIG. 9 as it would appear after the composition has beenrendered for HTML media. This media tree may be compared with the mediatree of FIG. 15 in which the composition is rendered to paper media. Inthis example of FIG. 16, the media tree 250 has a root HTML Page Set andthree child components, namely Page 0, Page 1 and Page 2. This mediatree for HTML media has three pages instead of the single page for papermedia because a composition to be rendered in HTML format has differentrequirements and a different layout as compared to that for paper media.In this example, the brochure composition of FIG. 9 would appear asshown in FIGS. 12a, 12b and 12c when rendered in HTML media. Hence, themedia tree of FIG. 16 has three different pages, Page 0, Page 1 and Page2. The number of pages used by a particular composition for a particularmedia layout is a matter of the content and the design choice for thatmedia, and the composition may take multiple forms for a particularmedia. For example, a property that may be explicitly set may controlwhether the number of pages is fixed or is allowed to vary with theamount of content. The layout of the brochure composition in HTML mediaas shown in FIGS. 12a, 12b and 12c, and as represented by the media treeof FIG. 16 is one such illustrative example of how a layout and a mediatree might appear for a composition.

The media tree of FIG. 16 may best be interpreted by reference back toFIGS. 12a, 12b and 12c. Page 0 of the media tree represents the layoutas shown in FIG. 12a, Page 1 represents FIG. 12b, and Page 2 representsFIG. 12c. Page 0 has seven children, namely, the title "Bass Products",the fish logo, "Rod Products" and "Net Products", "What's New","Brochure" and "Talk to Us". The first four children are shown withlayout coordinates in FIG. 16, while the last three are not shown, butwould have links as shown at 252. These children in the media treerepresent the items of content shown in FIG. 12a. As in the paper mediatree of FIG. 15, each content item has associated with it its extent, inthis case, its X and Y coordinates and its height and width.

Page 1 of the HTML media tree has the six children as shown whichrepresent the content from FIG. 12b. Likewise, each of these contentitems has an associated extent as shown below each content item in FIG.16. Similarly, Page 2 of the media tree has six children correspondingto the content items of FIG. 12c along with their associated extents ascan be seen in FIG. 16. As can be seen in Page 0, Page 1 and Page 2, thetitle and logo are repeated. An optimization may be to share contentelements by reference to a single instance. It should also be pointedout that new content items have been created for this HTML media becauseof the unique requirements for that media. For example, the new contentitems "Rod Products" and "Net Products" have been created and appearwithin Page 0. Likewise, the content item "Home Page" now appears onPage 1 and also on Page 2.

FIG. 17 shows at 260 the content item "Bass Products" from Page 0 inmore detail. The content item may be represented internally as an objector other similar data structure. Preferably, the content item isrepresented as a component. Shown in particular for the content item"Bass Products" is how that content item may have associated with itvarious properties that relate to different media. The function of thesedifferent properties for the different media is to allow the contentitem to be represented appropriately for a given media. For example,this content item currently has values for its HTML media propertiesbecause it is being rendered in HTML media as shown in FIG. 16.Accordingly, the properties associated with its paper media propertiesare left blank or are unused. Many other different types of propertiesunique to, or appropriate for HTML or paper media may be included withthis content item although they are not shown here.

Also shown for this content item "Bass Products" are a set of propertiesassociated with a potential rendering of the content item to VRML media.Shown are properties representing the content item's X, Y and Zcoordinates, and its height, width and depth. The additional propertiesof a Z coordinate and depth would be appropriate for a content itemrendered in VRML media because of the three-dimensional nature of thatmedia. Other properties that may be appropriate for this media arecolor, brightness, rotation and sound. For example, music sounds or evena video clip may be associated with a particular content item in acomposition, and these sounds and video may be associated with thecontent item via its properties. It should be appreciated that manyother different kinds of properties may be associated with a particularmedia type.

FIG. 18 at 270 shows in more detail the title and logo from Page 0 ofthe paper media tree of FIG. 15. Along with this portion of the mediatree, relevant portions of the content tree and design tree of FIGS. 13and 14 are also shown in order to demonstrate the relationship betweenthe three trees.

In this example, the root of the brochure composition is a CompositionComponent 271 that has a content facet, a design facet and a mediafacet. These three facets are the respective roots for each of thecontent, design and media trees. These three partial trees as shown inFIG. 18 are portions of the three trees that are shown in FIGS. 13, 14and 15. For example, the root of the content tree is Document Contentthat is linked to its child content item "Bass Products". The "BassProducts" content item is linked to the fish logo and is also linked tothe image of the fishing rod and to the image of the net (not shown) vialinks 276.

The root of the design tree is Document Design that has child componentNameplate and links 272 to the components Product List and Footer (notshown). As in FIG. 14, the Nameplate component has subcomponents Titleand Logo which have links to their respective content items that arealso shared with the content tree. The root of the media tree is thecomponent Paper Page Set which has one child component Page 0. Page 0has as children the title "Bass Products" and the fish logo as well aslinks 274 to the other content items that are shown in FIG. 15 but arenot shown here in FIG. 18. Thus, FIG. 18 illustrates how although thecontent, design and media trees may be viewed conceptually as separate,they may have a common root and common leaves, and at times may sharecommon intermediate components, although not in this example.

EXAMPLE OF ADDING CONTENT TO A COMPOSITION

An example of how the content tree, design tree and media trees aremanipulated when content is added to a composition is illustrated inFIGS. 19 through 22. These Figures illustrate graphically how contentmay be added to a composition and how each tree is changed accordingly,in one embodiment of the present invention. A more detailed discussionof the steps involved in this process is contained below in theflowcharts of FIGS. 29 through 36. In general, FIG. 19 shows contentthat is being dropped onto an existing composition, FIG. 20 shows thecontent, design and media trees before the drop, and FIG. 21 shows thesame trees after the content has been dropped onto the composition.

FIG. 19 illustrates one technique 280 for adding content to acomposition. Shown in FIG. 19 is a user interface 281 of a computersystem that includes a composition 282. Located within the composition282 are currently a photograph 284 and an empty text region 286. Theempty text region 286 may be represented by a particular text componentthat at this point has no context associated with it. In this example, auser wishes to drop content from a word processor 288 onto thecomposition 282. The word processor content 288 includes a title, afirst subheading with a body of text "xxx", and a second subheadinghaving two text items "yyy" and "zzz".

In a first step 290, this word processor content is imported into anembodiment of the present invention in the structure shown at 292. Theimport of this content may be performed in a wide variety of manners. Byway of example, the import of content may be performed by an objectbroker which is able to recognize, define and group content intoparticular objects.

In this example, the object broker has recognized the word processorcontent as structured text, and has grouped the content into a hierarchyof text paragraph components. For example, component text paragraph 296corresponds to the title of the content "WHO WAS KILROY?", textparagraph 298 corresponds to the first subheading "Here At One Time",and text paragraph 300 corresponds to the body of text "xxx".

This content may be dropped via a technique 294 onto the composition 282in any suitable manner. It is contemplated that this drop process may beperformed in a wide variety of manners. By way of example, content maybe dragged from a location on the user interface and dropped onto thelocation of the composition using any suitable pointing device, such asa mouse. Alternatively, the drop may be performed by the user doubleclicking a pointing device, or by the user identifying content inanother location within the composition in any other suitable manner.The content may also originate from any suitable source. By way ofexample, the content may appear within the user interface, the contentmay be identified by a file name, a URL address, a portion of adatabase, or any other similar means used to specify content using acomputer system. Additionally, content may be input automatically to acomputer system via a computer interface, a network connection, a modeminterface, an infrared or radio link interface, a sound recognitioninterface, a video interface, or any other similar means of inputtingdata into a computer system. Content may also be input via amail/messaging system, a data acquisition device, a telephone system,scanner, etc.

Turning now to FIGS. 20 and 21, FIG. 20 illustrates the internalrepresentation of the content, design and media trees before the contentis dropped, while FIG. 21 illustrates the representation of these treesafter the content has been dropped onto the composition.

FIG. 20 shows embodiments 310 of each of the content, design and mediatrees for the composition 282 of FIG. 19. The content tree 312 has aroot Document Content with one link to the only item of contentcurrently in the content tree which is the data 316 for the photograph284 of the composition. The design tree 314 has a root Document Designthat has one child component Product. Product has two subcomponentsProduct Illustration and Product Text, representing respectively placeholders for the photograph 284 of the composition and the empty textregion 286 of the composition. Product Illustration in this example hasone subcomponent Image. In this example, Product Illustration has nocaption, thus, there is no additional subcomponent for a caption locatedbelow the component Product Illustration in the design tree. The Imagecomponent of the design tree has various properties associated with itsuch as the quantity JPEG indicating the type of data, and a "no border"flag indicating that the photograph should have no border. A widevariety of other properties describing how the photograph should appearin this composition for a particular media are possible.

The Product Text subcomponent of Product in the design tree has onesubcomponent Text Flow which indicates a region for text and variousproperties for text that may be placed such as "structured" and "leftjustified". In this example, the Text Flow component indicates a regionfor text identified by the X and Y coordinates 55 and 30, and a widthand height of 40 and 35. At this point, there are no subcomponents toText Flow because there is no content in this region. It should beappreciated that a wide variety of other properties in addition to"structured" and "left justified" may be used in association with adesign component such as Text Flow to indicate how content should appearin a given media. Representative examples of other such properties aredescribed above.

The media tree 316 of FIG. 20 has a root Paper Page Set with onesubcomponent Page 0. On Page 0 is one subcomponent Product Rectanglewhich allocates media regions 318 and 319 for the Image 316 and the TextFlow component respectively. In other words, the media region defined bythe coordinates of the component Product Rectangle (namely 10, 30, 85,35) is divided into two regions, one of which includes the photograph,the second of which is currently an empty text region. At this point intime, the composition is ready for content to be dropped upon it. Oncethe content has been dropped onto the composition via a drop technique294 as shown in FIG. 19, then the content, design and media trees appearas shown in FIG. 21 according to one embodiment of the presentinvention.

FIG. 21 shows embodiments 330 of the content, design and media treesafter the content 288 of FIG. 19 has been dropped onto the composition282. As will be explained in detail below, each of the three trees hasbeen changed due to the dropped content. Starting with the design tree314', the component Text Flow has had numerous subcomponents addedbeneath it in the design tree hierarchy. The Text Paragraph componentsshown in a hierarchy 292 of FIG. 19 have been added in this samehierarchy as subcomponents to the Text Flow component. For example,below the Text Flow component is a single subcomponent Text Paragraphthat represents the title of the content, "WHO WAS KILROY?". Below thisfirst Text Paragraph component are two subcomponents, namely two moreText Paragraph subcomponents representing the first subheading "Here AtOne Time", and the second subheading "Missing In Action". In a similarfashion, these two Text Paragraph components have components below themin the design hierarchy representing their respective bodies of text"xxx", "yyy" and "zzz".

The content tree 312' still contains the root Document Content with alink to the Image 316, but now also has a link to the title of thecontent, "WHO WAS KILROY?". This title, in turn has content links belowit connecting to the first and second subheadings, which in turn havelinks to their respective bodies of text. For example, the secondsubheading "Missing In Action", has two content links, one to text body"yyy" and one to text body "zzz".

In a corresponding fashion, the media tree 316' now has additional linksdefining new regions for each of the new content items that have beenadded. The media tree still has links to the original media regions 318and 319 for the Image and Text Flow components respectively. Although,the positions for Text Flow has changed slightly reflecting the impactof the new content on fitting. Additionally, the Product Rectanglecomponent now has media links to each of the new Text Paragraphcomponents defining a specified media region within the ProductRectangle media region for each piece of new content. For example, theProduct Rectangle component now has a link to the Text Paragraphcomponent representing the body of text "xxx" that has media X and Ycoordinates of 70 and 45, and a width and height of 20 and 10respectively. In a similar fashion, the Product Rectangle mediacomponent has links to each of the other new Text Paragraph componentsdefining a particular media region for their respective content items asis shown in FIG. 21. The coordinates for the Product Rectangle have alsochanged from (10, 30, 85, 35) to (10, 30, 85, 55) because of the droppedcontent. Also, the height of Text Flow has changed from 35 to 55.Alternatively, a different design might require that the extents of eachof these components must remain the same, in which case differentattributes for the dropped text such as font and text size would bemodified in order to render the text in the same amount of space.

When content is dropped onto a composition (such as is shown in FIGS. 19through 21) an embodiment of the present invention makes use of acontent drop table 350 as shown in FIG. 22 in order to determine whethercontent may be dropped in a particular location, and if so, what type ofcomponent should be created for that dropped text. A content drop tablemay be implemented in a wide variety of manners. By way of example, thecontent drop table 350 of FIG. 22 is one such implementation. An exampleof a specific use of a content drop table by a receiving component of acomposition will be discussed in more detail below with reference tostep 512 of FIG. 29 and step 554 of FIG. 30.

In one embodiment of the present invention, a content drop table such asthe one shown in FIG. 22 exists for each component of a composition.Alternatively, the concept of a content drop table may be applied to agroup of components, or to a complete document. A component type in acomposition that may receive a drop is termed a receiving componenttype. The content drop table 350 has an identification section 352 and abody 353 that has columns 354 and 356. The identification section 352 ofthe content drop tables indicates for which receiving component type thecontent drop table applies. In this example, the content drop table 350is a drop table for a receiving component type of type Text Body.

The purpose of the content drop table is to attempt to match a potentialdropped content type with a possible appropriate subcomponent type forthe given receiving component type. In this example, a receivingcomponent of type Text Body has been designed to accept content ofeither type Text or OLE Object and map these specific content types to aparticular new subcomponent type. The drop table shows that droppedcontent of type Text may either match an In-line Heading component or anIn-line Body Paragraph component for a new subcomponent. A droppedcontent of type OLE Object will only match an In-line Illustrationsubcomponent. The first column 354 is labeled "Potential Dropped ContentTypes" because a dropped item of content may possibly be interpreted asdifferent types of content. The object adaptor that recognizes thisdropped content is able to identify the potential component types ofdropped content. For example, a short amount of text dropped onto acomposition might be interpreted as a type of caption, body text,heading, etc.

The content drop table may be used in the following manner. Assume thata word processor clip is dropped onto a composition. The UniversalObject Adaptor first constructs a list of the possible types such astext, OLE Object, etc. Next, the table is stepped through one row at atime. If the type listed in the left-hand column is not compatible withthe identified possible types of dropped content, then that row isskipped. However, if it is compatible, then an attempt is made toconstruct a component of a type as listed in the right hand column usingthe dropped content. If this attempt is successful, then the process isdone, if not successful, then the next row of the table is tried. It isalso possible for a user to explicitly choose which component type tocreate via a drop technique or other interaction with a user interface.It will be appreciated by those of skill in the art that other priorityschemes for interpreting and matching content types are possible. Theuse of this drop table will be discussed further with reference to FIGS.29 and 30 below.

MEDIA DIVISIONS

Another useful feature of the present invention is the concept of amedia division. A media division allows a particular region of a mediumto be split into other related regions according to a set of rules andproperties. The concept of a media division will now be discussed withreference to FIGS. 23 through 28. FIGS. 23 and 24 illustrate anembodiment of how a media division may be used within the presentinvention and FIGS. 25 through 28 provided illustrative examples ofvarious kinds of media divisions. The use of a media division in thecontext of an embodiment of the present invention will then be furtherexplained in more detail below with reference to FIGS. 34 and 35.

FIGS. 23a through 23d provides a simple product example to show amotivation for the use of a media division. FIG. 23a shows an exampleproduct layout 370 for a particular product that may appear in acomposition 372 as shown. This example product has a picture region onthe left side of the composition and a name and body text on the rightside of the region. Within the picture region is the image itself alongwith a caption and a cutline both below the image. On the right side ofthe composition, the body text appears below the name of the product.

FIG. 23b shows an embodiment of the design tree 380 representing thedesign hierarchy for the example product of FIG. 23a. The top levelcomponent Product has three children, namely Picture, Name and Body. Theintermediate component Picture contains three subcomponents, namely theImage, the Caption and the Cutline. It should be noted that the designtree 380 provides a hierarchical break down of the organization of theexample product, but is not required to provide information as to thelayout of each individual component. On the other hand, the media tree390 of FIG. 23c provides more detailed information about the layout ofeach of the components within the design tree by way of properties ofthe media facet of each component. Although not shown, each component ofthe media tree has an associated extent defined by various properties,indicating the position within the media where the component is located.

The media tree 390 of FIG. 23c may be thought of as inherentlyrepresenting the geometric containment of regions within regions. Inother words, each node or component of the media tree corresponds to aregion on a page (for paper page media) or to some other region for adifferent media. The media tree 390 has a root Product with two childrenPicture and Text Area. The Picture component indicates the left handside of the composition in which the picture information will appear.The Text Area component indicates the right hand side of the compositionin which the appropriate text will occur. The Text Area component hastwo children Name and Body. The Picture component has two subcomponents,Image and Picture Text Area, while Picture Text Area has childrenCaption and Cutline. In a geometric sense, the component Picture hasbeen divided horizontally into two regions by a virtue of its having thetwo children Image and Picture Text Area. This can be seen from theexample product of FIG. 23a in which the image appears above the text ofthe picture (including the caption text and cutline text). Byconvention, a region that is divided by a horizontal line into an upperpart and a lower part is termed a "horizontal" division, while a regiondivided by a vertical line into parts side by side is termed a"vertical" division.

This horizontal division of the region for Picture may be contrastedwith the vertical division of the overall Product component. That is,between the Picture component (representing the picture region on theleft hand side) and the Text Area component (representing the textregion on the right hand side) there is a vertical division ofinformation between the picture and the text area as can be seen fromFIG. 23a. In FIG. 23a, the picture information appears on the left handside of the composition while the text area contains textual informationthat appears on the right hand side of the composition. Also, thecomponent Picture Text Area has two subcomponents Caption and Cutline.Because Caption and Cutline appear one above the other, the picture textarea region has been divided horizontally. Similarly, the Text Areacomponent is divided horizontally into a Name component and a Bodycomponent. An embodiment of a technique for specifying how these variouscomponents are divided either vertically, horizontally or otherwise willbe discussed below with reference to FIG. 24.

In this simple example, both the design tree and the media tree appearfairly similar. However, in a more complex composition, the design treeand media tree may be quite different. For example, consider the bodytext of FIG. 23a. Currently this body text only appears on one page.Thus, the body text may be contained in one media region of the page andonly one media component is needed in FIG. 23c to represent the locationof the body text on the one page. However, if this body text were toextend into a second page, or to be split across columns, this body textwould be represented by different media components in the media tree.While at the same time, the design tree would still represent the bodytext by a single component Body. In this case, the design and mediatrees would appear differently. Also, it should be appreciated thatother more complex representations of information in a design tree and amedia tree are possible. In another example, in an HTML version of abrochure with a home page having various products on subsequent pages,the live link present on the home page is part of the products bydesign, but is part of the first page by media (i.e., far from otherproduct components).

For illustrative purposes, the design tree of FIG. 23b and the mediatree of FIG. 23c are presented as separate trees. However, it should benoted that these trees have various of the components in common. Forexample, the components Product, Picture, Name, Body, Image, Caption andCutline are components that are common to both trees. Accordingly, thedesign and media trees of FIGS. 23b and 23c may be more appropriatelyviewed in a single, yet more complex tree structure of FIG. 23d. In thisexample, it may be seen how various components are shared between thetwo trees, while at the same time, the media tree has additionalcomponents such as Text Area and Picture Text Area. For example, becausethe name and body of the example product of FIG. 23a occur together in aparticular media region, they are represented below a media componentText Area within the media tree below the component Product. Thisadditional media component helps to define a media region for both thename and the body. In a similar fashion, the picture of FIG. 23a may beviewed in a geometric sense as being divided into an upper media regionrepresenting the image, and a lower media region representing the textfor the picture which in this case is the caption and the cutline. Thus,in the media tree both the caption and the cutline of FIG. 23a arecontained within a particular media region that is appropriatelydesignated a Picture Text Area in the media tree.

An embodiment of a technique for representing the concept of dividing amedia region is shown in FIG. 24. FIG. 24 shows in more detail the mediatree 390 of FIG. 23c along with the corresponding media divisions thatassist in the division of the media for its components. The media treeuses a media division at each component in order to decide how to dividethe available region for that component among the children of thatcomponent. In other words, each parent component which represents aparticular region, or extent, is responsible for placing its childrenwithin that region or in some relationship to that region.

A media division may be viewed as a set of properties of a particularcomponent describing a set of rules that dictate a layout for thechildren of that component. A media division may include any number ofdifferent properties and may divide up an extent of a component in avariety of manners. By way of example, specific media divisions areillustrated in FIG. 24, and particular types of media divisions will bedescribed below with reference to FIGS. 25 through FIG. 28. Theproperties, rules and values for a media division may be represented asbeing associated with a particular component in any suitable manner. Byway of example, a media division may be represented as a separate objectthat is linked to a particular component. The media tree 390 of FIG. 24illustrates how the components Product, Text Area, Picture and PictureText Area each have an associated media division. The use of thesedifferent media divisions will now be explained.

The media division 392 associated with the Product component may haveany number of properties associated with it that help define how themedia region for Product may be divided amongst the children of Product.Media division 392 shows various properties and values associated withthe Product component. By way of example, media division 392 has aproperty with a value "two-way" indicating that the media region forProduct should be split into two pieces for two different children. Thevalue "vertical" indicates that the media region for Product should besplit by a vertical line into a left side and a right side as can beseen in FIG. 23a. The value "fit 25%" indicates that the first child onthe left hand side of the region allocated for Product component shouldbe placed into an absolute measurement of 25% of this region. The valueof "margin 12 points" indicates that a margin 12 points wide willseparate the region for each child. The property "swap" with the valueof "false" indicates that the fit value of 25% should be measured fromthe left hand edge of the parent region and not the right hand edge.

The component Text Area has a media division 396 that is also a two-waydivision thus providing two regions for components Name and the Bodywithin the text region of their parent component. Unlike media division392, media division 396 is a horizontal division meaning that ahorizontal line divides text area into a top portion and a bottomportion for the text of the name and the body respectively. This mediadivision also has an "exact fit" property value indicating that the nameand the body text should not be allocated to a particular percentage orabsolute measured portion of the region of the parent, but that a firstchild (for example, Name) should be allocated as much space as it needs,and that the remaining space should be allocated to the second child,for example, Body Text.

The component Picture has a media division 394 also indicating that itsregion should be split in a horizontal fashion into two portions for itstwo children Image and Picture Text Area. The component Picture TextArea has a media division 398 which also splits the region allocated forit horizontally into two portions for its children, Caption and Cutline.

It should be appreciated that many other types of properties and valuesmay be included in a media division to specify how a child may be placedwithin the extent of the parent or to specify how a child may be placedin relationship to the parent. It is not always required that a child beplaced within the region of the parent; the child may also be placedoutside the region allocated to the parent. By way of example, the fitproperty for a media division may be an absolute fit or exact fit. Anabsolute fit may give a percentage of the region to be allocated to thechild, may give specific media units indicating how much region shouldbe occupied, or may give an aspect ratio for the child. An exact fitvalue in a media division indicates that the first child placed is givenas much of the region as it requires and the rest of the region is leftover for the second and subsequent children. Other properties that areuseful in a two-way media division include bottom-up versus a top-downfit, the type of balance (e.g., absolute, exact, negotiated), optimalplace for a dividing line, minimum or maximum acceptable dividing line,acceptability of a dividing line away from optimal, options for treatingtext/content of a child that flows to a different page (e.g., keepsecond child on initial page, move second child to end of flow onanother page), no page break, break page if forced, always break page,etc.

FIGS. 25 through FIG. 28 illustrate various types of media divisionsthat may be used in embodiments of the present invention. It iscontemplated that other kinds and forms of media divisions are possible.FIGS. 25a-25f illustrate examples 400 of a one-way media division. Aone-way media division is used when a parent component has only onechild component and the parent component is attempting to place thatchild component in some relationship to the region occupied by theparent component.

FIG. 25a is an example of a child component 402 being placed in theinterior of a region occupied by a parent component 404. Such a one-waymedia division may place the child in relationship to the parent bymeasuring a margin from each of the edges of the parent region. FIG. 25bis an example of a child component 406 being placed in a comer region ofthe parent component 408. FIG. 25c is an example of a child component410 being placed along the edge of a region occupied by the parentcomponent 412. FIG. 25d is an example of a child component 414 beingplaced in a location that overlaps with the region occupied by theparent component 416. FIG. 25e is an example of a child component 418placed within the region occupied by the parent component 420 yet in adifferent orientation to that of the parent. FIG. 25f is an example of achild component 422 being placed at a distance from the region occupiedby the parent component 424. A child component may also coincidecompletely with the region occupied by the parent component, or mayoccupy any portion of the region of the parent component, such as aleft-hand side or a right-hand side.

FIGS. 26a-26h illustrate examples 430 of a two-way media division. Atwo-way media division may be used when two children components must beplaced in some relationship to the region occupied by the parentcomponent. A two-way media division may also appear as shown in FIGS.25a through 25f in which one child component corresponds to the childcomponent shown in FIG. 25, and the second child component correspondsto the parent region of FIG. 25. The two child components of a two-waymedia division may also overlap one another in any fashion irrespectiveof the location of the parent region. FIG. 26g is one possible exampleof two child regions overlapping.

FIG. 26a is an example of a vertical two-way media division in whichchild 1 occupies a left hand region of the parent region 432 and child 2occupies a right hand region. FIG. 26b is an example of a horizontaltwo-way media division in which child 1 occupies a top portion of theparent region 434 and child 2 occupies a lower portion. FIG. 26c is anexample of an absolute fit two-way media division in which an absolutevalue of 33% indicates a region 436 for child 1 and the remaining region437 is left over for child 2. FIG. 26d is similar to FIG. 26c expect inthis example the swap property has been set to "true" indicating thatthe 33% absolute value should be measured from the right hand edge. Inthis example then, child 2 occupies 33% of the parent region 442 (region440) while child 1 occupies the left over region 441. FIG. 26e is anexample of a horizontal division that has a margin between a child 1 anda child 2 inside a parent region 444. FIG. 26f is an example of atwo-way media division 446 in which child 2 is placed adjacent to child1; in this example, child 1 coincides with the parent region. FIG. 26gis an example of a two-way media division 448 in which child 1 overlapsthe region occupied by child 2. FIG. 26h is an example of a two-waymedia division 450 in which child 1 is placed apart from the regionoccupied by child 2; here, child 1 coincides with the parent region.

FIGS. 27a-27e illustrate examples 460 of three-way media divisions. FIG.27a is a first example of how a parent region 462 may be divided amongsta child 1, a child 2 and a child 3. FIG. 27b is a second example of howa parent region 464 may be divided amongst three children. FIG. 27c is athird example of how a parent region 466 may be divided amongst itsthree children. FIG. 27d is a fourth example of how a parent region 468may be divided amongst three children. FIG. 27e is a fifth example ofhow a parent region 470 may be divided between three children in anon-rectangular fashion. It should be appreciated that for any mediadivision, the region occupied by a child may be specified in any manner,and not necessarily by a rectangular coordinate system. For example, atwo-way media division may split a parent region by an angular or curvedline, and not necessarily by a horizontal or vertical line.

Also, a three-way media division may appear as any of the two-way mediadivisions in which a third rectangle (or other shape) is placed in somerelationship to the other two child regions, such as inside, next to, oroverlapping. Also, a three-way media division may only specify therelationship of the three child regions irrespective of the location ofthe parent region. For example, a three-way media division may specify arelationship in which three child regions are adjacent in some manner,are overlapping, or are each separated by a certain amount of space.

FIGS. 28a-28e illustrate examples 480 of an N-way media division. AnN-way media division indicates a region of media that is being allocatedfor any number of children. In a preferred embodiment, an N-way mediadivision refers to allocation of a parent media region using horizontaldivisions. For example, FIG. 28a is an example of an N-way mediadivision in which the parent region 482 is being divided into absolutepercentages for each of its children (which may be of any number). FIG.28b is an example of a top-down exact fit allocation in which the parentregion 484 is being filled by its four children in an exact fit fashionand the extra media is left over at the bottom of the parent region 484.FIG. 28c is another example of an exact fit media division in which theparent region 486 is filled by exactly fitting its three children intothe region and the extra space in the region is divided up equally andinterspersed before and after each child. FIG. 28d is another example ofan exact fit media division in which the three children are fit exactlyinto the space they require in the parent region 488 and any extra spacein the region is inserted between the children. FIG. 28e is an exampleof an abstract parent region 490 that is being divided up into anynumber of regions for any number of children that the parent may have.It should be appreciated that a parent region may take many forms andmay be divided up in any fashion, including allocating a child regionoutside of the parent region.

Many other types of media divisions are possible. For example, athree-dimensional medium may be divided up into three-dimensional units.A media made up of sound or video may be divided up into time units ofsound or video. Also, a media division that represents a whole page of amedia is also contemplated.

PROCEDURAL EMBODIMENTS OF THE PRESENT INVENTION

In addition to the tree representations, media divisions and otherembodiments of the present invention described above, additionalembodiments of the present invention relate to techniques forautomatically adjusting the content, design and media of a composition.For example, if a user adds content in the form of a text or a picture(for example) to a composition, the present invention is able toautomatically fit that content to the media by either adjusting thecontent, changing the design, or reformatting the media layout for thecontent. Also, for a given content in a composition, the user may alsochoose a new design for that content and the present invention willautomatically reformat the content in that new design. And furthermore,if the user has specified a particular content in a specified design,the user may utilize embodiments of the present invention in order torender that content into different media such as screen media, paperpage media, HTML media or other.

The following FIGS. 29 through 38 relate to various embodiments andtechniques for performing this automatic fitting of particular contenthaving a specified design into a particular media. FIG. 29 describes anembodiment in which new content is added to a composition and thepresent invention is able to automatically fit that content to themedia. FIG. 37 relates to an embodiment in which the user chooses a newdesign for a particular composition and FIG. 38 relates to an embodimentin which the user wishes to render particular content into a new medium.The flowcharts for these embodiments will now be discussed.

FIG. 29 describes an embodiment of a technique for adding new content toa composition. The composition to which the content is being added maybe a document shell, such as a blank design form, or may be acomposition that is in the process of being built, or a composition thatis already completely filled with content such as text and images. In afirst step 504 a piece of content is dropped on a component of thecomposition. For example, the user may indicate a location within thecomposition to which the content is to be dropped via drag and drop orother technique. This location typically corresponds to a receivingcomponent of the design tree of the composition. For example, a user maydrop a photograph upon a blank outline of a region for an image in acomposition.

In a next step 508 a universal object adaptor identifies potentialobject types of the content that was dropped. The content that isdropped may be interpreted as a single object with multiple types, ormay be an aggregate of different types within a single object. A widevariety of potential object types are possible. By way of example,standard types may be text, hypertext, database records,objects/applets/components, audio/video, three-dimensional models (e.g.,VRML), real-time data, images, tables, etc. Other types are possible,for example, text may be plain text, rich text, bitmap, etc. A picturemay be a bitmap, a vector drawing, GIF, JPEG, or other. Also, a givencontent object may possibly be interpreted as being of differentpotential types. For example, an image is typically of just one type,"image", whereas a text paragraph of content may be of type "text", abit map type, a meta file, etc.

Next, in step 512 an attempt is made to match one of the potentialcontent object types with a type accepted by the receiving component.This step will return a result from this matching attempt which isprocessed below in step 516. This matching attempt is used to determineif a receiving component can handle the type of the dropped contentobject. Each receiving component uses a content drop table that includesthe list of types that the receiving component may receive in apreferred order. This content drop table is discussed above withreference to FIG. 22. This matching step is also able to place droppedcontent in a correct location if it was dropped on an incorrect type.For example, if a receiving component may receive pictures or text and aproduct is dropped upon this component, it may decide to treat thedropped product as text. Also, if text is dropped on a picture that onlycan receive images, this text may appear below the picture as a caption.The process by which this matching attempt occurs will be discussedbelow with reference to FIG. 30.

Other situations in which content is placed at a position that does notinclude the drop point may be handled according to a design choice. Forexample, in a brochure design, dropping a picture upon a picture mayalways replace the content completely. In a report design, dropping aparagraph upon an existing paragraph may insert the dropped paragraph atthe drop point. As will be explained below with reference to FIG. 30, ifa receiving component cannot accept a drop, the media facet container ofthe receiving component is searched in order to find an appropriatecomponent. For example, if a picture is dropped upon text in a brochure,the text may not accept it. The product containing that text will acceptthe picture by way of replacing the content of the existing picture forthat product. Also, if a complete product is dropped upon text, the textcomponent will not accept it, and the existing product containing thattext will not accept the dropped product either. But, the document willaccept the dropped product and will insert a new product component inaddition to other existing products. In addition, in a report design,for example, a dropped picture on text may be accepted, unlike theexample given above. That is, in a report design, a text component mayaccept a dropped picture and create an in-line illustration component.It should be appreciated that other protocols and design choicesregarding how to treat content dropped at various locations may bepossible.

Once the match attempt of step 512 finishes, the result of this matchattempt is processed in step 516. Four examples of possible results fromthis match attempt are shown in step 516. Other results are alsopossible. A reject result may occur if an attempt is made to dropcontent onto a component that cannot accept any potential object typesof the content. In this scenario, the system may respond by indicatingto the user that a drop may not occur, or may accept the drop and thenreject it in some fashion and indicate this result to the user. In oneembodiment for a reject result, a warning icon may appear over an itemto be dropped when the item is positioned over a receiving componentthat cannot process the drop. This feedback may happen in real time asthe user is dragging content over a composition.

A second result might be that a child component needs to be created fromthe receiving component. This may occur if structured text or other textis being added to a region of text already in place. A third result mayindicate that the content of the receiving component may need to bechanged. This may occur if the user attempts to drop a new piece ofcontent wholesale over an existing piece of content with the intentionof replacing the existing content. A fourth result may be that variousproperties of the receiving component may need to be changed. This mayoccur if a new font is dropped upon a receiving component already havingtext in a different font. In this case, the font properties of thereceiving component would need to be changed. In various of the resultsof step 516 the design tree may be changed by either adding or deletingcomponents. This step 516 will be discussed in more detail below withreference to FIG. 31.

In step 520 the current content of the composition is fit to theindicated media. This step takes the existing content that the user hasspecified within a particular design, and will adjust the content in asuitable fashion in order to fit it to the required media region. Thisstep will be discussed in more detail below with reference to FIG. 33.This step may involve changing the overall size and form of the media ifdesign properties permit this change. For example, the number of totalpages may be fixed for a brochure, but may be able to be expanded for areport design.

Next, in step 524 the entire composition including the content tree, thedesign tree and the media tree is rendered to the specified media. Forexample, if the specified media is screen media, then the system willrender the composition for viewing on the computer screen. Likewise, ifthe specified media is paper page media, the system will print ontopaper pages, but may also allow the paper page media to be viewed on acomputer screen via a print preview option. In a similar fashion, acomposition rendered to HTML media may be formatted for presentation ina Web page format and translated into the HTML language, although thesesame Web pages may be prepared for viewing on a computer screen via aprint preview option. A composition may be rendered by taking the extentfor each content element along with information from the design tree forthat content element and outputting this information into the desiredmedia. For example, a composition may be rendered to paper media byusing standard drivers of a computer system to output the information.

A composition may be rendered to media in a wide variety of manners. Byway of example, rendering to paper media may be done by using operatingsystem and/or device API's and drivers. Rendering to a computer screenmay also be done by using operating system and/or device API's anddrivers. Rendering to HTML format may be done by using macros in mediafacet prototypes, and by using publicly available HTML specifications.Rendering to an OLE object may done by reference to published OLEinterfaces. Rendering to other applets may also be done by reference topublished specifications.

FIG. 30 describes the match content step 512 of FIG. 29. Once thecontent object has been dropped upon a receiving component, this steptries to match a component type in the composition to the droppedcontent. For example, the user may drop a content object in an incorrectlocation, or upon a receiving component that cannot accept that content.This step will make use of the content drop table of FIG. 22 in order toassist in matching the dropped content with a receiving component. Instep 554 a search is made through the receiving component's content droptable to find a first match with the prioritized list of newsubcomponent types for one of the potential content object types for thedropped content.

For example, referring back to the content drop table of FIG. 22, if thedropped content type has a potential type of "text", the highestpriority new subcomponent type of In-line Body Paragraph will matchfirst and indicate that a new subcomponent type of In-line BodyParagraph should be created. However, if the dropped content is only oftype "image", then the drop table will first try to match In-line BodyParagraph and In-line Heading to the content type, will fail, and willthen match In-line Illustration with "image". Typically, a sequentialsearch is done through the content drop table to find a preferred match.It is preferable to match a high priority table entry with any of thepotential object types. In other words, the table will attempt to matchthe highest priority content drop table entry with any of the potentialobject types before attempting to match a lower priority table entrywith any of the potential object types.

Step 558 test whether a match has been found in the content drop table.If a match has been found, then in step 562 the result associated withthis match from the content drop table is retrieved and returned to step516 of FIG. 29 for processing. The result from the content drop tableindicates one of the possible results of step 516 along with a new typeof component to be created if necessary. It should be appreciated thatmany other types of results may be associated with a match in thecontent drop table and may be processed as well in step 516. A resultmight also be implemented by specialized code associated with a specificcomponent or component type.

On the other hand, if a match is not found in step 558 then controlmoves to step 566. For example, if the user attempts to drop a largepicture onto a tiny caption there will be no match. Step 566 attempts tofind a receiving component that may match the type of the droppedcontent object by working up the media facet tree. If no media facetparent exists for the receiving component, then in step 570 a rejectresult is returned to be processed by step 516 of FIG. 29. However, if amedia facet parent does exist, then in step 574 these potential contentobject types are passed to the media facet parent of the receivingcomponent to see if a match can be found with either that media facetparent or with one of its other children. An example of this process maybe illustrated with reference to FIG. 23c. For example, if text isdropped upon the Image component, step 558 may not find a match, butthen would pass the dropped text up to the parent media component whichis Picture. Picture in turn may then pass the dropped text down to oneof its children such as Caption or Cutline for the text to be droppedinto that position. In this fashion, a dropped content object may berouted to a correct location even if it is dropped in the wrong spot. Inthe case where Picture does not yet have a Caption child, the font ofthe dropped text may become a parameter of the construction of aCaption.

Once the potential content object types have been passed to the mediaparent (which is then treated as the receiving component), the processloops back up to step 554 in order to search the content drop table forthis new receiving component in order to find a possible match for thedropped content. The flow continues in this fashion until one of thepossible results is returned to step 516. Because this procedure loops,all parents in the media facet tree will be checked for a match untilthere is either a match or a reject result.

FIG. 31 describes the process result of match attempt step 516 of FIG.29 in more detail. The step receives a result from the match attemptprevious step and processes that result. A first step 604 determineswhether the result was to reject the drop attempt. If the answer is yes,then in step 608 the drop is either ignored or disabled in some fashionand then the procedure of FIG. 29 ends. The drop may be ignored if theuser had been allowed to perform the drop or the drop may be disabled inreal time as the user is attempting to drop content upon a composition.

Step 612 tests whether the result is "create a child component". If thatis the result then in step 616 a child component for the receivingcomponent is created using the dropped content object to provide thecontent for that new child component and also using the type suppliedfrom the drop table. In this situation, because new content is beingadded to a new location, it is considered a "place" operation. Forexample, if text is dropped upon a picture, a new caption may be createdinstead of the text replacing the picture in the composition. A newcomponent may be created in any suitable manner. By way of example, aconstruction procedure is associated with each component type. To createa new component, the factory for that component is found, the componentis created and the dropped content is associated with the new component.The new component may also inherit information and properties from itsparent object as well. In a preferred embodiment the creation method ishandled by the prototype of the component. Step 616 will be discussed inmore detail below with reference to FIG. 32 as a place operation. Oncethis step 616 concludes, control returns to step 520 of FIG. 29.

Step 620 tests whether the result is "change content of the receivingcomponent". If so, then in step 624 the dropped content object is passedto the existing content object of the receiving component to handle itas a content edit. In this example, a new child component is not beingcreated, but the content associated with the receiving component isbeing changed, or replaced. In other words, the old content may beremoved and replaced with the new content. Various types of edits may bepossible in this step. For example, the old content may be replacedentirely, the new content may be inserted into the old content, or thenew content may be concatenated with the old content. This step will bediscussed in more detail below with reference to FIG. 32 as a "replace"operation. Once this step 624 concludes, control returns to step 520 ofFIG. 29.

Step 628 tests whether the result is "change properties of the receivingcomponent". If so, then in step 632 the new property name-value pairsfrom the dropped content object are retrieved. This situation may occur,for example, when a new font is being dropped upon an existing contentobject. In another example, if a font is dropped upon a component of aproduct list type its subcomponents will inherit that new font. This isbecause a product list component type may not have any text of its ownbut its subcomponents will. That is, the subcomponents will haveassociated text, but may not have a text face property of their own. Inaddition, the retrieved property name value pairs may simply specify newvalues for existing properties or may also define new properties andvalues for the existing content object. It may also be that one propertyof the content is being changed or that many properties are beingchanged.

Next, in step 640 the current properties of the receiving component arechanged to the retrieved properties and values from the dropped contentobject. This operation may be performed by matching the retrieved valueswith the corresponding properties of the receiving component. Forexample, if a new font "Times" is being dropped onto an existing contentobject that has a font "Helvetica", then the retrieved property font ismatched with the existing property font and the new value "Times"replaces the old property value "Helvetica".

It is possible that setting new property values may cause a change inthe design tree. That is, a design change might be associated with aparticular setting in a property value. For example, a document mightinclude page numbers and a styling property that indicates whether thesepage numbers will appear or not. If the styling property is changed toindicate that page numbers should not be included with the document,then the design tree representing this document would be changed toremove the components representing the page numbers from the document.Other properties and their values may similarly require a change in thedesign tree. If a property value indicates a change in the design tree,then the design tree is modified accordingly at this point. Once step640 has concluded control returns to step 520 in FIG. 29. If the answerto step 628 is no, then other results may be processed at this pointbefore control returns to step 520.

FIG. 32 describes the place/replace step called from either step 616 orstep 624 of FIG. 31. The place/replace operation is used to either placenew content into a composition or to replace existing content with newcontent according to a particular editing operation. Step 654 determineswhether this is a place or a replace operation. A place operation isused when a new component needs to be created, whereas a replaceoperation is used typically when an existing component may be reused. Anexisting component may be reused because it is desired to preserve itsproperties, or a portion of its existing content may be reused. If thisis a place operation, then in step 662 a new component is allocated. Anew component will be allocated using the type matched in the drop tablefrom step 554 of FIG. 30.

If this is a replace operation, then in step 658 the system determineswhether there is already an existing subcomponent of the new componentthat has a type consistent with that matched in the drop table. BecauseFIG. 32 is a looping operation that will create as many new componentsas necessary, the new component referred to in step 658 refers to notonly the initial receiving component the first time through the loop,but any reused components in further iterations of the loop. Anyexisting subcomponent that is consistent with the type matched in thedrop table will be suitable. The match does not have to be the highestpriority type in the drop table, but a higher priority match ispreferable. If there is a choice between a replace or a place operation,in a preferred embodiment a replace is preferred.

An example of an existing subcomponent that might be reused may bedescribed as follows. Consider a document having a title where the titleappears in a unique font style designed by the user. The user may bereplacing the document but wishes to keep the title the same. However,it may be that the new replacing document has a title in a very plainfont style. Because the user has previously changed the title to aunique font style, it would be desirable to keep the title in the uniquefont style. In this example, it would be desirable to reuse the oldtitle (or component) in order to retain the unique font style. Thus, theexisting subcomponent representing this title in a unique font would bereused. However, it may be a matter of design choice whether the userwishes to always retain old font styles, or to replace existing fontstyles with a new font style.

If an existing subcomponent with a type that matches is not found instep 658, then in step 662 a new component is allocated. However, ifthere is an existing subcomponent that may be reused, then in step 666the variable new component is set equal to this subcomponent to bereused. Once either step 662 or step 666 has concluded, it is thennecessary to modify each of the design, content and media trees in orderto insert this new component (whether the new component is newly createdor whether it is a reused component).

In step 670 the design tree parent pointer of this new component is setto point to the original component that received the dropped contentobject. In this fashion, the new component is made a subcomponent of theoriginal receiving component. If this is not the first iteration throughthe loop procedure of FIG. 32, then the design parent pointer of thisnew component is set to point to its immediate parent (which may havebeen created in a previous iteration of the loop). Thus, the design treeis modified to incorporate the new component. Next, in step 674 thecontent tree parent pointer of the new component is set to point to thedropped content object. If this is not the first iteration through thelooping procedure of FIG. 32, then the content parent pointer of the newcomponent would be set to a next content subobject of the droppedcontent object. In terms of a containment hierarchy, the new componentis contained in the dropped content object. In this fashion, the contenttree is updated to include the dropped content object (or a next contentsubobject).

Next, in step 678 the media tree parent pointer of the new component isset to null and the complete media facet of the new component is set tono allocation. The media parent pointer is set to null in order to allowthe design facet parent to determine or compute where the new componentappears in the layout, while the media facet is not allocated. Theattempt to render the new component will then trigger a re-layoutresulting in allocation of space for the new component. Once the threetrees have been updated, the new component is initialized. In step 682an initialization procedure for the newly allocated component isexecuted. This initialization procedure will be executed for a componentthat has been allocated but not for a component that has been reused.Typically, an initialization procedure sets the properties of thecomponent to their default values.

At this point in the looping procedure of FIG. 32, the procedure ispoised to perform a walk of all the substructure included within thedropped content object in order to create components to match these subobjects. If the dropped content object includes no substructure, thenthis place/replace operation is done and control returns to step 520 ofFIG. 29. However, if step 686 determines that the dropped content objectdoes include substructure, then the place/replace operation will performa looping operation in order to allocate or reuse a component for eachof the sub objects within the dropped content object. For example, for asimple image there will be no substructure and the operation will end.However, for structured content 80 such as shown in FIG. 6, a newcomponent will be allocated or reused for each of the title, subheadingsand the body text. Structured text is one example of a content objectwith substructure.

If there is substructure, in step 690 the next content subobject of thedropped content object is retrieved. Next, in step 694 the place/replaceoperation is called again by looping back up to step 654. The nextcontent subobject retrieved will be passed into the place/replaceoperation as the next content object, and the new component (which iseither a newly allocated component or a reused component) will be passedinto the place/replace operation in the place of the receivingcomponent.

The example of FIGS. 19 through 21 may be used to illustrate thissituation of FIG. 32. When the imported word processor content 292 isfirst dropped upon the empty text region 286, the Text Flow component ofFIG. 21 is considered the receiving component and the newly createdcomponent is a Text Paragraph representing the title "WHO WAS KILROY?"On a second iteration through the place/replace operation, the nextcontent subobject of the dropped content will be the first subheading"Here At One Time" and the component taking the place of the receivingcomponent would be the component that had just been allocated, namelythe Text Paragraph component representing the title "WHO WAS KILROY?" Inthis fashion, the looping procedure of FIG. 32 is able to place all ofthe structured content found in the word processor content 288 of FIG.19 into its appropriate place in the design, content and media trees.When all of the sub objects of the dropped content have been placed orreplaced, this looping procedure ends after step 686, step 516 of FIG.29 is done and control returns to step 520.

FIG. 33 now describes the fit content to media step 520 of FIG. 29. Atthis point in FIG. 29, new content has been added to the composition.But because most media is limited in some sense (i.e., paper media maybe limited to a certain number of pages), it may be necessary to eitherreadjust the content, the design or the media of the composition inorder to make the information fit within the media. The information maybe adjusted to fit within the desired media in a wide variety ofmanners. By way of example, the design may be readjusted to allow thecontent to fit within the available media. Alternatively, the media maybe adjusted, i.e. the media may be lengthened, in order to allow thecontent to fit within the media. That is, the number of pages in adocument may be changed, or the calculation of the media divisions maydictate a different layout. Also, the content itself may be adjusted insome fashion to fit within a defined media region. For example, a fillerimage may be clipped from a larger stock image in order to makeeverything fit within a page. It should be appreciated that any onethese techniques may be used by itself, or in conjunction with the othertechniques. Step 520 describes a method for fitting the information of acomposition into a media region by adjusting the content and design.

Once new content has been added to a composition, there may be a varietyof reasons why the overall content of the composition must be adjustedin order to fit it within the available media. For example, if a page oftext is nearly full and a picture is dropped onto this page, the designtree will be changed because a new component is created for the picture.In this situation, not only are the design tree and the content treechanged, but also the media tree will have new regions calculated forthe newly dropped picture. In another example, if only additional textis dropped onto a page of text, the design tree would not be changedbecause the existing text is merely being replaced and no new componentswould need to be created in the design tree. However, if the addition ofnew text causes the body of text to overflow onto a second page, then anew page is created and a new text component for that new page iscreated within the design tree. In addition, the media tree willcalculate a region for the newly added text on the second page.Similarly, an added picture to a page of text may push text onto asecond page. In all of these situations, either one, two or three of thetrees have changed, and the layout must be readjusted in order to fitthe available media.

In general, the step of fitting content to media involves the layout ofeach component element. Fitting content to media starts by walkingthrough the design tree from top to bottom and creating new mediacomponents between existing components in the design tree where needed.In an optimization, only a subtree may need to be computed. Tentativeregions are calculated for existing media components on the way downthrough the trees, and as the recursive algorithm pops back up throughthe trees, these tentative regions are replaced by final regions. If aparticular layout does not fit, then global scale factors may be used toadjust each component type.

In a first step 704 all of the content scale factors are initialized tonominal values. A content scale factor indicates how a particular typeof content may be scaled either up or down in order to assist in fittingall of the content into the available media. A wide variety of scalefactors may be used. In one embodiment, one scale factor is used foreach type of content. In other words, each type of content (e.g., textor graphics) may be adjusted by a particular factor in order to eitherenlarge all content items of that content type, or to reduce all contentitems of that content type. It is also contemplated that there may be acontent scale factor corresponding to each item of content or that ascale factor may apply to any permutation of content items and contenttypes. Also, a scale factor may refer instead to particular designcomponents, or particular regions within a media, in addition toaffecting the scaling of content by type. In a preferred embodiment,there is a content scale factor for content of type text, a scale factorfor content of type graphics, an opaque content scale factor (forobjects such as OLE embedded objects), a white space scale factor, anoverlap scale factor and a layout pressure scale factor. Initially,these content scale factors are set to a nominal value, for example,each scale factor is set to 100%.

Once the content scale factors have been initialized, in step 708 themedia tree is computed starting from the root of the composition. Themedia tree is computed by passing to it the design hierarchy embodied inthe design tree and walding through this design tree in order to createnew media components between design components (if necessary) and tocalculate media regions for each of the existing and new mediacomponents. This step 708 is a recursive step which will call itself foreach component in the design tree. Once this step has completed, eachitem of content will have an associated extent, or defined region, thatindicates how much space within the available media the content uses.This step 708 will be explained in more detail below with reference toFIG. 34.

The compute media tree step 708 also returns overall rich extentinformation including the combined extents for all descendants in themedia tree. Once this information has been returned, the system willknow if the document fits overall or not into the available media, andwill also know if any particular component does not fit. That is, theinformation returned will also indicate whether a particular leafcomponent does not fit, and if not, by how much it does not fit.

Next, step 712 determines whether the content fits closely enough intothe available media. If the content does fit then control moves to step524 of FIG. 29 and no adjusting of the content is required. However, ifthe content does not fit, then the content may be adjusted in order forit to fit within the available media. If the content is over theavailable media (an over fit) then the content is defined as notfitting. If the content is under the available media (an under fit),then the content will fit, unless a particular design calls for thecontent filling as much of the available media as possible. In thissituation, the content may be enlarged to fit more closely in theavailable media. Also, certain thresholds may be used in order to definethe meaning of "closely enough". A "leftover" medium threshold may bedefined indicating how much extra medium is acceptable if the content isunder fit. A "time" threshold may also be used to indicate for how longor for how many times the process has attempted to fit the content tothe media. Other thresholds may also be used. If the content does notfit, then flow moves on to step 716.

In step 716 the fit culprits for each content type are computed for theoverall composition. In other words, this step determines which types ofcontent contribute to the overall over fit, or to the overall under fit.For each type of content contributing to the non-fit, a value is givenindicating by how much that content type contributes (such as apercentage or in absolute units). A fit culprit is determined byreference to the media division of each media component. For example, ifa picture placed next to a corresponding body of text occupies morevertical space then the text, then the picture is determined to be thefit culprit for that particular localized region. On the other hand, ifa region of text is much longer than a picture placed next to it thenthe region of text is determined to be the local fit culprit because itcontributes more to the over fit of the composition than the picture.Therefore, for a vertical media division typically only one of thecomponents within this vertical division will be determined to be thefit culprit.

For a horizontal media division (where a region is divided by ahorizontal line), it may be that one or all of the components within thehorizontal division are fit culprits. For example, in a horizontal mediadivision that uses an absolute fit, if a body of text does notcompletely fill an absolute region, it is not a fit culprit because itis not filling the whole region. However, another body of text thatfills an absolute region completely will be determined to be a fitculprit. Likewise, for a horizontal division that uses exact fit, all ofthe components that are fit exactly into this region will be consideredfit culprits because they fill the region completely and do not leaveany extra space.

All of these fit culprits are calculated at a local level. However, itmay be the case that a child is a local fit culprit but if the parent isnot a fit culprit then the local fit culprit is irrelevant. For example,in a vertical division if a group of pictures occupies a left handportion of the division and text occupies the right hand portion, anyone of the group of pictures may be determined to be a local fit culpriton the left hand side. But if overall, the group of pictures occupiesless vertical space than the corresponding body of text on the righthand side, then the text is considered the fit culprit for that regionand not the group of pictures.

In one embodiment, this step is performed by walking the media tree andby looking at the media divisions to determine local fit culprits anddetermining whether the local fit culprits may be bubbled up to a higherlevel. The result of this step is a value that indicates how much eachtype of content is responsible for the over fit (or the under fit). Forexample, for a single page paper media where the content overflows byfive inches, the result of this step may return values indicating thatoverall the text content overflows by two inches and that the graphiccontent overflows by three inches. Once an amount of over fit or underfit is calculated for each content type then control moves to step 720.

In step 720 the content scale factors for each content type arerecomputed. This step may change one, many or all of the content scalefactors. The purpose of this step is to adjust the content either upwardor downward in order to optimize the resulting content so it will fitwithin the available media. In one sense, the compute media tree step708 may be viewed as somewhat of a black box into which are input thevarious content scale factors. Step 720 adjusts these scale factors in aparticular fashion in order to achieve a different result from thecompute media tree step 708. The desired result is a close enough fit aspossible, as determined by step 712. Step 720 takes into account variousinputs such as the over fit or under fit of each content type, how theclose the current fit is, how close a previous fit attempt was, theprevious content scale factors used and the current content scalefactors. Step 720 may be performed in a variety of manners. This stepmay be performed by a function maximizer for nonlinear discontinuousfunctions. By way of example, this step may be performed by using asimulated annealing technique, or other techniques such as geneticalgorithms. One technique for performing this step will be discussed inmore detail below with reference to FIG. 36.

Once the content scale factors have been recomputed for each contenttype in step 720, then the procedure of FIG. 33 loops back to step 708in which the media tree is again computed from the composition rootusing the new content scale factors. The procedure continues looping inthis fashion until step 712 determines that the content does fit closelyenough to the available media.

FIG. 34 describes the compute media tree step 708 of FIG. 33. This stepwill be called recursively for each component in the design tree as theprocedure walks down through the design tree. As the design tree iswalked, the context of where a particular component is locatedgeographically on page and where it occurs in the design hierarchy willaffect the layout options for that component.

A first step 754 tests whether the current component is a primitive.This test is used as a terminating condition for this recursiveprocedure. A component is a primitive (or atomic) typically if it is aleaf node containing content such as an image, text or data. If thecomponent is a primitive, then in step 758 the component is measured andits extent is returned. Thus, as the design tree is recursively walked,when a leaf node is reached its extent will be returned back up to itsparent in order that the parent may calculate its own extent.

The extent of a component is the N-dimensional space in the media thatis occupied by the component. For example, for a two-dimensional mediasuch as a paper page, the extent of a component is its X and Ycoordinates and a width and a height. For VRML media, the extent of acomponent may be the above and may also include its depth andorientation. Thus, the extent of a component includes a variety ofinformation and depends upon the media to which the content is to berendered. The extent information returned for a component also includesinformation about whether a component fits in its allocation region andif not, by how much it is over fit or under fit. The extent of acomponent is typically dependent upon properties such as "fixed width"and "fixed height". For example, if a property of an image dictates thatthe image must have a fixed width, then based upon that property theimage will adjust itself in order to fit in a particular region. Thatis, the image will keep its aspect ratio and will automatically adjustits height in order to fit within the region. Other properties thataffect a component's extent are: tolerance to aspect ratio change,visual embellishments such as borders, frames, etc., orientation,inter-character, inter-line or inter paragraph spacing of text, and auser specification (such as interactively assigning a greater or lessersize).

Next, in step 762 the procedure continues with a depth first walk of thedesign tree by identifying all design children of the current component.As this is a recursive procedure, this step will be performed for eachnode in the design tree. At this step, only the immediate designchildren of the current component are identified in order that new mediacomponents may be created between the current component and itsimmediate design children as will be explained below in step 770.

Next, in step 766 a tentative layout option is selected based upon theuser design choice, the content of the composition, aspects of thedesign tree, etc. For a particular design choice, many different layoutoptions may be possible. This tentative layout option is based upon avariety of factors. For example, the user design choice is the initialdesign that the user has chosen for this composition. This initialdesign choice will influence which layout option is chosen. In addition,the content of the composition may dictate a particular layout option.For example, an extremely tall picture, a very wide picture or anextremely small amount of text may indicate that a particular layoutoption is more suitable for that particular content. Also, properties ofthe parent component may indicate a particular layout option. Forexample, a property of a parent component may dictate that any childpicture be made as wide as possible.

Other aspects of the content may also influence the layout option. Forexample, new products may have been added to a particular composition,or columns may have to be changed from columns to rows if too manycolumns exist. Also, a very small caption and cutline may beinappropriate to the right of a very tall image and may be moreappropriately placed below the image in order make the design moresymmetrical. This step of selecting a tentative layout option may beperformed automatically by the system by choosing an optimal option.Alternatively, the user may be presented with any number of layoutoptions and may have a choice in selecting a particular option.

Next, in step 770 new media components are created as needed between thecurrent component and all immediate design children of the currentcomponent. The purpose of this step is to create portions of the mediatree between design components by creating new media components andforming links between these new media components and their parents andchildren. An example of a new media component between design componentsmay be seen in FIG. 23d. In this Figure, it can be seen that the toplevel design component Product has design children components Picture,Name and Body. However, the media tree for this composition does notdirectly form a link between the top level component Product and itschildren Name and Body. Because a media region is to be allocated on theright hand side of the composition, a new media component labeled TextArea is created between the Product component and its design childrencomponents Name and Body.

Once these new media components have been created in step 770, in step774 all of the required media tree links between the current componentthrough the new media components to the design children of the currentcomponent are formed. For example, once the new media component TextArea has been created as shown in FIG. 23d, it is linked to its parentcomponent Product and to its children Name and Body. In this fashion,the media tree is created and built up through the use of the mediafacets while walking down the design tree.

Once these new media components have been created and linked, in step778 tentative layout properties which include media division propertiesare assigned to the current component and to the new media components.These tentative layout properties are based in part upon the tentativelayout option that was selected in step 766. The media divisionproperties assigned may include choosing a media division type (such asone-way, two-way, or other) and setting any of the media divisionproperties such as "vertical" or "horizontal", "fit", or "swap". Thestep of assigning tentative layout properties includes setting not onlythese media division properties but also includes setting any otherproperties of the current component that affects the layout of itschildren.

Once these tentative layout properties for a current component and itschildren media components have been assigned, it is necessary to solvefor the media division properties on the current component. In otherwords, the extent for each child needs to be calculated in order tocalculate the extent of the current component. This step 782 of solvingfor the media division on the current component may be performed indifferent manners. By way of example, one technique for performing thisstep will be discussed in more detail below with reference to FIG. 35.The result of performing step 782 will be to return the extent of thecurrent component. The purpose of this step is to look at the mediacomponent children of the current component and to assign to them aparticular media division (or region) according to the layout propertiesassigned to the current component. By solving for the media division onthe current component, this process determines a layout between thecurrent component and its design children. But because the layout of thedesign children may in part be determined upon how their children arelaid out, this step will end up calling the procedure "compute mediatree" (FIG. 34) recursively.

Once step 782 has completed, step 786 determines if this overall layoutis suitable. If the layout is suitable, then control returns to step 712of FIG. 33 in order to determine if the content fits closely enough inthe available media. However, if the layout is determined not to besuitable, then this procedure loops back up to step 766 in order tochoose another tentative layout and to create and link new mediacomponents as necessary.

The layout may be determined to be suitable or not suitable in step 786according to a wide variety of factors. By way of example, variousfactors may help to determine suitability and each one may be weighteddifferently. These factors include: the suitability of a particularlayout option for the chosen design, how well the extents of thecomponents fit in the available media, if the content fits with theoriginal design intent, properties of the media divisions of thecomponent, the number of alternative layout options that have beentried, the time that has been spent in trying to choose an appropriatelayout option, the suitability of the layout option for the particularmedia, and how a particular layout option ranks amongst all of thelayout options considered. Other factors may also be considered in orderto determine layout suitability.

FIG. 35 describes the solve media division step 782 of FIG. 34. Thisfigure describes one embodiment of a technique for solving for the mediadivision. This step takes a current component and calculates a mediadivision only for its direct children. In other words, this is aspecific numerical algorithm for solving for the properties of a mediadivision. Other techniques may be utilized that make use of the abstractexpression of the relationship between the parent component and itschildren components.

In a first step 804 each direct media child of the current component isdetermined. In step 808 a tentative media region is assigned to eachmedia child based upon the layout properties of the current componentand also based upon the media region allocated to the current component.A tentative media region is assigned because this region may be adjustedbelow in step 816. In one embodiment, media children are assigned aparticular rectangular media region having a particular relationship tothe current component. For example, in a two-way vertical mediadivision, one child will be assigned a left-hand portion of the currentcomponent's media region and the second child will be assigned theright-hand portion.

Once a media region has been assigned to each media child, then in step812 the procedure "compute media tree" of FIG. 34 is called for eachmedia child component. This procedure will be recursively called andreturns the extent for each child. This recursive call is done in adepth first fashion. In this fashion, tentative media regions areassigned to media children as the procedure walks down through thedesign tree, and when the procedure recursively pops back up through thedesign tree the actual extents for each child are returned (ascalculated in the terminating step 758 of FIG. 34). Once these actualextents for each child are returned, then final media regions may beassigned to a particular media component. In step 816 final mediaregions are assigned to each media child of the current component basedupon the returned extent for each child.

Step 820 tests whether the constraints provided by the media division ofthe current component are satisfied. These media division constraintsmust be checked again because in step 816 the media region for eachchild may have been changed. For example, if an image was too big forits allocated space and it was given more space in which to fit, it maythen be necessary to recalculate space for the other components below orabove the image. In another example, if an address for a companyletterhead is placed as a footer at the bottom of the page, it may havebeen placed higher than expected. If so, then regions above this footermay have to be recomputed. If the constraints are not satisfied then theprocedure loops back up to step 808 in order to assign a new tentativemedia region for each media child.

If, however, the constraints are satisfied, then in step 824 the mediaextents from all of the direct media children are combined in order todetermine the extent of the current component. In other words, once theregions occupied by the children of the current component have beendetermined, it is then possible to determine the overall region that isoccupied by the current component. For example, if a product regioncontains a picture and body text, it is first necessary to determine theamount of space needed by the picture and the body text in order todetermine how much space must be allocated to the overall product. Oncethe extent of the current component has been determined, then this stepof solving the media division for that current component has concludedand control returns to step 786 of FIG. 34.

This step 824 includes combining all of the extents from all of thedirect media children of the current component. And because each mediachild includes knowledge of the extents from its media children, allthis rich extent information will be combined as this procedure popsback up recursively. In other words, knowledge of the extents from alldescendants is used in order to determine the extent of the currentcomponent. This rich extent information includes information aboutwhether a child fits within a given region in addition to informationabout the size of that child. Thus, the rich extent information for thecurrent component includes knowledge as to whether the region allocatedfor the current component is enough or not.

FIG. 36 describes the recompute content scale factors step 720 of FIG.33. The purpose of this step is to take into account the relativeculpability of each content type that may contribute to an over fit oran under fit of the content and to rescale the content scale factors foreach type in order to achieve a better fit. In a first step 854 variousparameters are input to this procedure. These parameters include thecurrent content scale factors, the current overall fit (over fit orunder fit and by how much), the rich extent information for allcomponents, the total culpability for each content type (e.g., thenumber of excess inches for each type), the previous scale factors used,and the previous fit results. Next, in step 858 the current fit iscompared to the optimal fit. If the current fit is too small for theavailable media then control moves to step 862, while if the current fitis too big then control moves to step 866.

Based upon the result of step 862 the content scale factors will beadjusted in different manners. For example, if the current fit is toosmall and the previous fit was also too small, this indicates that thecontent scale factors were adjusted upward previously but not by enough.Thus, in step 870 the content scale factors will be adjusted upwardagain by a preset amount. This preset amount may be any suitable numberfor adjusting all of the scale factors or for adjusting each oneindividually. In one optimization, if the underfit is 90% of the regionor more, then an informed guess may be used based upon the inputparameters in order to adjust the scale factors upward by other than apreset amount. Other heuristics may also be used to adjust these scalefactors. For example, by design choice, images may be shrunk more thantext if the content needs to be reduced while text may be enlarged morethan images if the content needs to be enlarged. In one embodiment, textand graphics scale factors may also be adjusted separately to avoidunnecessary shrinkage of either text or graphics, depending on thepercent of overall fit contributed by text content and by graphicscontent. In one embodiment, a formula may be used to determine a newtext scale factor and a new graphic scale factor. For example, the newtext scale factor=(Percent Text)/((Percent Text)² +(Percent Graphics)²),while the new graphics scale factor=(Percent Graphics)/((Percent Text)²+(Percent Graphics)²). In this formula, Percent Text and PercentGraphics refer to the percentage of each of these content thatcontribute to the overall fit (how much of a fit culprit they each are).

On the other hand, if the current fit is too small but the previous fitwas too large, this indicates that the content scale factors werepreviously reduced but were reduced by too much. In this case, in step874 the content scale factors are adjusted back upward to halfway to theprevious scale factors. After either steps 870 or 874 has concludedcontrol returns to step 708 of FIG. 33.

Steps 866, 878 and 882 may be performed in a similar fashion the steps862, 870 and 874. For example, based upon the result of step 866 thecontent scale factors will be adjusted in different manners. Forexample, if the current fit is too big and the previous fit was also toobig, this indicates that the content scale factors were adjusteddownward previously but not by enough. Thus, in step 878 the contentscale factors will be adjusted downward again by a preset amount. Thesefactors may be adjusted downward in a fashion similar to that describedfor step 870.

On the other hand, if the current fit is too big but the previous fitwas too small, this indicates that the content scale factors werepreviously increased but were increased by too much. In this case, instep 882 the content scale factors are adjusted back downward to halfwayto the previous scale factors. After either steps 878 or 882 hasconcluded control returns to step 708 of FIG. 33.

FIGS. 29 through 36 have described an embodiment of a technique foradding new content to a composition. FIG. 37 now describes a techniquefor choosing a new design for a composition while the content and themedia may stay the same. For example, one design for a document mayinclude a table of contents while another design may not. If a newdesign is chosen that requires a table of contents, the new design willhave a table of contents component, a new title page component and linksbetween the table of contents and the rest of the document. It should beappreciated that this change of design may take place independently of achange of content or of media, or may take place in conjunction with achange in content and a change in media.

This procedure of choosing a new design for a composition begins withchoosing a new design in step 902. This new design may be chosen bymodifying a current design using the user interface, or the user mayselect a design from a catalogue of designs within the system. Forexample, a user may choose a vertical format for a brochure instead ofthe current horizontal format. Alternatively, in other embodiments, thesystem may automatically select a design for the user based upon variouscriteria.

In step 904 both the content tree and the design tree representations ofthe composition are saved to a temporary location within the system.These trees may be saved in any suitable fashion as will be appreciatedby one of skill in the art. Thus, in this fashion, the content elementsare saved along with their corresponding components which also allowaccess to the design properties for that content element.

Next, in step 908 the new design description chosen by the user isreceived from the system catalogue. The designs in the system catalogueof designs may be standard designs or may be custom designs that a userhas created previously and stored. The new design description mayinclude a root with a number of components, or may simply be a rootcomponent. This new design may appear differently due because ofdifferent properties or because of different drop tables used. Forexample, the root of this new design may have different propertiesdefined upon it that will be inherited by all of its subcomponents.Similarly, any components initially attached to the root may haveparticular properties and values assigned to them. Also, the root (orany of its components) may utilize a different content drop table thanthat used by the old design. In this way, a new design may appeardifferently from an old design.

Next in step 912 the design tree of the composition is restored usingthe new design description. In this step, the new design description isused to form the new design tree. That is, the root of the new designtree (and any subcomponents of that root) are incorporated into thecomposition as the current design tree. At this point, a design treeexists for the composition (although it may just be a root), but nocontent has been added.

Once the design tree of the composition has been restored, in step 916the saved content from the saved content tree is placed back into therestored composition. This saved content may be restored to thecomposition in any suitable manner. By way of example, the saved contentmay be added to the new design tree by a simulation of dropping content.That is, the saved content may be dropped onto the composition whichcontains the new design tree. This dropping of content onto acomposition may be performed as discussed in FIGS. 29 through 32.Additionally, an optimization may be performed in order to preserve olduser document settings. For example, override properties of a particularcomponent in the old design tree may be applied to a new componentassociated with the same content.

This optimization may be performed in the following manner. As contentis being dropped onto the new design, a new component may be created forthat content. For example, a product from a brochure may have a newcomponent created for it. And because a particular item of content isassociated with its design component from the old design tree, the olddesign properties of this component (applied to the item of content) areavailable for inspection. By way of example, content may be associatedwith design tree components as illustrated in FIGS. 6, 18, 21 or byusing other techniques. In other words, a walk of the content tree (ascontent is being dropped) still allows the old design components andproperties to be accessed. If the new component created (product) is ofthe same type as the old component (product), then all of the overrideproperties for that old component will be transferred over to the newcomponent. In this fashion, local property overrides for a component (asapplied to a particular item of content) are preserved.

Consider the brochure example in which the user is switching from ahorizontal format for products to a vertical format. The first productmay have an emphasis property set locally which overrides the documentproperties and presents that product all in bold (instead of in regularfont like the rest of the document). When the old content is droppedonto the new vertical design, the new document may only specify regularfont. When the first product is dropped, it is noted that its newcomponent (product) is the same as its old component (product). Includedwithin the properties defined upon the old component are the property"horizontal" and the property "bold". Because "horizontal" is a propertyinherited from the document and not locally defined, it will not becopied over to the new component. But, "bold" is a locally defined,override property. It will be copied over to the new component, thuspreserving the first product as being all in bold in the new verticalformat. At this point, the content and design trees now exist for thenew composition.

Next, in step 920 the content of the new design is fit to theappropriate media by calling the fit content to media step of FIG. 33.This step will walk through the new design tree and create the mediatree and assign regions to each content item. This step of fittingcontent to media has been discussed in greater detail above. Once thisstep has concluded, this procedure for choosing a new design for acomposition is complete.

In addition to adding new content to a composition or changing thedesign of a composition, a user may wish to render a composition into adifferent medium. FIG. 38 describes an embodiment of a procedure forperforming this function. For example, if the user has produced acomposition rendered for paper media, the user may also wish to publishthis same document in an HTML format, or vice-versa. The composition maybe rendered to a new medium independent of any change to the content orto the design. Alternatively, it is also possible to render acomposition to a new medium if the content and/or the design is alsochanging.

In a first step 954 a new medium is identified. This new medium may beidentified by the user issuing a command to the user interface or it mayalso be identified automatically by a computer system when it isdesirable to publish to a new medium. A wide variety of new media may beidentified. By way of example, these new media may be paper, HTML, anOLE object, VRML, or any of the media previously discussed. Next, instep 958 a new media tree root object is constructed based upon the newchosen medium. This step is a pre-programmed function supplied as amethod of a built-in factory object.

At this point, step 962 calls the fit content to media procedure of FIG.33 above. This fit content to media procedure will walk through thedesign tree of the composition and create the new media tree and assignlayout regions for each content item. This step utilizes the existingcontent and design trees and will recompute a new media tree fromscratch. The old media tree will be invalidated globally. This fitcontent to media step has been discussed in detail above. Once step 962has completed, in step 966 the composition is rendered to the newmedium. Step 966 may be performed as discussed above in step 524 of FIG.29.

Embodiments of the present invention as described above employs variousprocess steps involving data stored in computer systems. These steps arethose requiring physical manipulation of physical quantities. Usually,though not necessarily, these quantities take the form of electrical ormagnetic signals capable of being stored, transferred, combined,compared, and otherwise manipulated. It is sometimes convenient,principally for reasons of common usage, to refer to these signals asbits, values, elements, variables, characters, data structures, or thelike. It should be remembered, however, that all of these and similarterms are to be associated with the appropriate physical quantities andare merely convenient labels applied to these quantities.

Further, the manipulations performed are often referred to in terms suchas identifying, running, or comparing. In any of the operationsdescribed herein that form part of the present invention theseoperations are machine operations. Useful machines for performing theoperations of embodiments of the present invention include generalpurpose digital computers or other similar devices. In all cases, thereshould be borne in mind the distinction between the method of operationsin operating a computer and the method of computation itself.Embodiments of the present invention relate to method steps foroperating a computer in processing electrical or other physical signalsto generate other desired physical signals.

Embodiments of the present invention also relate to an apparatus forperforming these operations. This apparatus may be specially constructedfor the required purposes, or it may be a general purpose computerselectively activated or reconfigured by a computer program stored inthe computer. The processes presented herein are not inherently relatedto any particular computer or other apparatus. In particular, variousgeneral purpose machines may be used with programs written in accordancewith the teachings herein, or it may be more convenient to construct amore specialized apparatus to perform the required method steps. Therequired structure for a variety of these machines will appear from thedescription given above.

In addition, embodiments of the present invention further relate tocomputer-readable materials that include program instructions forperforming various computer-implemented operations. The materials andprogram instructions may be those specially designed and constructed forthe purposes of the present invention, or they may be of the kind wellknown and available to those having skill in the computer software arts.Examples of computer-readable materials include, but are not limited to,magnetic materials such as hard disks, floppy disks, and magnetic tape;optical materials such as CD-ROM disks; magneto-optical materials suchas floptical disks; and hardware devices that are specially configuredto store and perform program instructions, such as read-only memorydevices (ROM) and random access memory (RAM). Examples of programinstructions include both machine code, such as produced by a compiler,and files containing higher level code that may be executed by thecomputer using an interpreter.

FIG. 39 illustrates a typical computer system in accordance with anembodiment of the present invention. The computer system 100 includesany number of processors 102 (also referred to as central processingunits, or CPUs) that are coupled to storage devices including primarystorage 106 (typically a random access memory, or RAM), primary storage104 (typically a read only memory, or ROM). As is well known in the art,primary storage 104 acts to transfer data and instructionsuni-directionally to the CPU and primary storage 106 is used typicallyto transfer data and instructions in a bi-directional manner. Both ofthese primary storage devices may include any suitable of thecomputer-readable materials described above. A mass storage device 108is also coupled bi-directionally to CPU 102 and provides additional datastorage capacity and may include any of the computer-readable materialsdescribed above. The mass storage device 108 may be used to storeprograms, data and the like and is typically a secondary storage unitsuch as a hard disk that is slower than primary storage. It will beappreciated that the information retained within the mass storage device108, may, in appropriate cases, be incorporated in standard fashion aspart of primary storage 106 as virtual memory. A specific mass storagedevice such as a CD-ROM 114 may also pass data uni-directionally to theCPU.

CPU 102 is also coupled to an interface 110 that includes one or moreinput/output devices such as such as video monitors, track balls, mice,keyboards, microphones, touch-sensitive displays, transducer cardreaders, magnetic or paper tape readers, tablets, styluses, voice orhandwriting recognizers, or other well-known input devices such as, ofcourse, other computers. Finally, CPU 102 optionally may be coupled to acomputer or telecommunications network using a network connection asshown generally at 112. With such a network connection, it iscontemplated that the CPU might receive information from the network, ormight output information to the network in the course of performing theabove-described method steps. The above-described devices and materialswill be familiar to those of skill in the computer hardware and softwarearts.

Although the foregoing invention has been described in some detail forpurposes of clarity of understanding, it will be apparent that certainchanges and modifications may be practiced within the scope of theappended claims. For instance, the embodiments of the present inventiondescribed are applicable to other types of application programs asidefrom the one described above. Also, the design engine of the presentinvention may be based upon any suitable technology, and not necessarilyupon the use of a facets engine. A composition may contain many kinds ofcomponents, the types of components described herein are examples oftypes that may be used. Also, the content, design and media aspects of acomposition may represent a wide variety of information. An example ofdifferent types and functions of content, design and media is shown inFIG. 3. Shown in the top half of FIG. 3 are examples of various types ofcontent, design and media. Shown in the bottom half of the Figure areexamples of various functions performed by content, design and media.

Although particular structures have been shown for representing andconnecting the content, design and media trees of the present invention,a composition of information may be separated symbolically into content,design and media aspects using other representations andinterconnections. In particular, content elements may be associated withdesign components in different ways. The above description has alsoprovided illustrative examples of how content may be added, a design maybe changed or a new medium specified. It should be appreciated thatother more complex modifications are also be possible. Also, a givencontent may be fit to a particular media by also adjusting the design orby adjusting the media.

An embodiment of a media division has been described above showing inparticular various properties. This media division may be represented indifferent manners, and may be associated with a parent component and itschildren using different techniques. Also, a media division may includemany other properties that describe the layout of children components inrelation to a parent component. Also, solving for the media divisionproperties of a current component in order to lay out its children maybe performed in different manners. Once a layout has been generated,this layout may be modified, for example, by adjusting scale factors forthe composition. Other techniques for modifying a layout may also beused in order to fit content to a medium. Therefore, the describedembodiments should be taken as illustrative and not restrictive, and theinvention should not be limited to the details given herein but shouldbe defined by the following claims and their full scope of equivalents.

We claim:
 1. A computer-implemented method of adding content to acomposition having a chosen design and automatically calculating alayout for said composition, said method comprising:receiving a contentobject dropped upon a receiving component of said composition, saiddropped content object having at least one potential type, saidcomposition represented by a plurality of components in a design treeand including a plurality of content elements associated with saidplurality of components; matching a potential type of said contentobject with a compatible type of said receiving component; modifyingsaid design tree to incorporate said content object; and calculatingmedia layout values for each content element of said plurality ofcontent elements of said composition using said design tree such thatsaid plurality of content elements are laid out in said chosen design.2. A method as recited in claim 1 wherein said matching a potential typeof said content object with a compatible type of said receivingcomponent is performed in part by using a content drop table.
 3. Amethod as recited in claim 1 wherein said matching a potential type ofsaid content object with a compatible type of said receiving componentincludes:determining whether a match is found; and wherein when it isdetermined that a match is not found, passing said potential types ofsaid content object to parent components of said receiving component todetermine whether a match is found with a compatible type of any of saidparent components.
 4. A method as recited in claim 1 wherein saidmodifying said design tree to incorporate said content objectincludes:creating a content component to represent said content object;and inserting said content component into said design,tree as asub-component of said receiving component.
 5. A method as recited inclaim 1 wherein said calculating media layout values uses a media treerepresentative of a medium in which said composition may be rendered,said media tree having media components associated with said medialayout values for each content element.
 6. A computer-implemented methodof adding content to a composition having a chosen design andautomatically calculating a layout for said composition, said methodcomprising:receiving a content object dropped upon a receiving componentof said composition, said dropped content object having at least onepotential type, said composition represented by a plurality ofcomponents in a design tree and including a plurality of contentelements associated with said plurality of components; matching a typeof said dropped content object with a compatible type of said receivingcomponent; changing the content of said receiving component based uponthe content of said dropped content object; and calculating media layoutvalues for each content element of said plurality of content elements ofsaid composition using the design tree such that said plurality ofcontent elements are laid out in said chosen design.
 7. A method asrecited in claim 6 wherein said matching a potential type of saidcontent object with a compatible type of said receiving component isperformed in part by using a content drop table.
 8. A method as recitedin claim 6 wherein said matching a potential type of said content objectwith a compatible type of said receiving component includes:determiningwhether a match is found; and wherein when it is determined that a matchis not found, passing said potential types of said content object toparent components of said receiving component to determine whether amatch is found with a compatible type of any of said parent components.9. A method as recited in claim 6 wherein said calculating media layoutvalues uses a media tree representative of a medium in which saidcomposition may be rendered, said media tree having media componentsassociated with said media layout values for each content element.
 10. Acomputer-implemented method of adding content to a composition having achosen design and automatically calculating a layout for saidcomposition, said method comprising:receiving a content object droppedupon a receiving component of said composition, said dropped contentobject having at least one potential type, said composition representedby a plurality of components in a design tree and including a pluralityof content elements associated with said plurality of components;matching a type of said dropped content object with a compatible type ofsaid receiving component; changing properties of said receivingcomponent based upon properties of said dropped content object; andcalculating media layout values for each content element of saidplurality of content elements of said composition using the design treesuch that said plurality of content elements are laid out in said chosendesign.
 11. A method as recited in claim 10 wherein said matching apotential type of said content object with a compatible type of saidreceiving component is performed in part by using a content drop table.12. A method as recited in claim 10 wherein said matching a potentialtype of said content object with a compatible type of said receivingcomponent includes:determining whether a match is found; and whereinwhen it is determined that a match is not found, passing said potentialtypes of said content object to parent components of said receivingcomponent to determine whether a match is found with a compatible typeof any of said parent components.
 13. A method as recited in claim 10wherein said calculating media layout values uses a media treerepresentative of a medium in which said composition may be rendered,said media tree having media components associated with said medialayout values for each content element.
 14. A computer-implementedmethod of adding content to a composition having a chosen design andautomatically calculating a layout for said composition, said methodcomprising:receiving a content object dropped upon a receiving componentof said composition, said dropped content object having at least onepotential type, said composition represented by a plurality ofcomponents in a design tree and including a plurality of contentelements associated with said plurality of components; determiningwhether a compatible type of said receiving component matches with apotential type of said dropped content object, wherein when it isdetermined that no compatible type of said receiving component matcheswith a potential type of said content object, performing the followingsteps,identifying a parent component of said receiving component,determining whether a potential type of said dropped content objectmatches with a compatible type of said parent component, and whereinwhen it is determined that a potential type of said dropped contentobject matches with a compatible type of said parent component,calculating media layout values for each content element of saidplurality of content elements of said composition using the design treesuch that said plurality of content elements are laid out in said chosendesign.
 15. A method as recited in claim 14 wherein said determiningwhether a compatible type of said receiving component matches with apotential type of said dropped content object is performed in part byusing a content drop table.
 16. A method as recited in claim 14 whereinsaid calculating media layout values uses a media tree representative ofa medium in which said composition may be rendered, said media treehaving media components associated with said media layout values foreach content element.
 17. A method as recited in claim 14 wherein whenit is determined that a potential type of said dropped content objectmatches with a compatible type of said parent component, said methodfurther comprises modifying said design tree to incorporate said contentobject.
 18. A computer-implemented method of recognizing content addedto a composition having a chosen design, said methodcomprising:receiving a content object dropped upon a receiving componentof said composition, said dropped content object having at least onepotential content type, said composition represented by a plurality ofcomponents in a design tree and including a plurality of contentelements associated with said plurality of components; identifying saidat least one potential content types of said dropped content object;creating a component having an associated content element representativeof said content object and of one of said identified potential contenttypes; modifying said design tree to incorporate said created component;and placing said content element within said composition such that saidcontent element appears in said composition in the context of saidchosen design.
 19. A method as recited in claim 18 wherein said creatinga component utilizes a content drop table in order to determine anappropriate component to be created.
 20. A method as recited in claim 18further comprising:calculating media layout values for each contentelement of said plurality of content elements of said composition usingsaid design tree such that said plurality of content elements are laidout in said chosen design.